uery = query;
>>
>> }
>>
>>
>>
>> }
>>
>>
>>
>> *Properties.xml*
>>
>>
>>
>>
>>
>>
>> dt.application.CountryNameScan.operator..prop.databaseUrl
>>
>> {databaseUrl}
>
gt; How would I read the custom properties in my operators? , are these
> properties available to all the operators which run on different nodes of
> the cluster?
>
>
>
> Regards,
>
> Surya Vamshi
>
>
>
> *From:* Munagala Ramanath [mailto:r...@datatorrent.com]
>
You can add multiple XML configuration files at *src/site/conf* in your
project and select one of them at launch time.
This is described briefly at
http://docs.datatorrent.com/application_packages/ under the section entitled
*Adding pre-set configurations*
Ram
On Tue, May 17, 2016 at 6:43 AM, Muk
Take a look at (in Malhar):
contrib/src/main/java/com/datatorrent/contrib/hbase/HBasePOJOPutOperator.java
It doesn't have a full application but has sample uses of the HBase output
operators. Some of the other
test files in the same directory have sample applications run using
LocalMode, for examp
Take a look at the "Operator Properties" section of
http://docs.datatorrent.com/application_packages/
If a similar syntax works for the enum case, can you add it to that
document and submit a PR ?
Thanks.
Ram
On Mon, May 9, 2016 at 5:05 AM, Yogi Devendra
wrote:
> Hi,
>
> I want specify propert
, so the goal
> was to remove that stream and the mongo operator and create a new operator
> for the console and stream from jsonMapping to the console.
>
>
>
> *From:* Munagala Ramanath [mailto:r...@datatorrent.com]
> *Sent:* Wednesday, May 04, 2016 10:55 AM
> *To:* users@
I just launched an application on the sandbox and had no such problems. I
connected to the app with:
connect application_1462344831356_0004
and was able to see the logical plan with:
show-logical-plan
Can you tell us more about your environment, which Hadoop distribution,
which version of Apex, e
How about using
wildcards: dt.application.myapplication.operator.*.prop.myproperty
Ram
On Wed, Apr 27, 2016 at 11:51 PM, Yogi Devendra
wrote:
> Hi,
>
> I have a requirement where I need to share some property across multiple
> operators. As such, this is an application level property.
>
> Somet
Is the tuple itself not aware of the key fields ? I'm thinking about the
file in Malhar:
*demos/machinedata/src/main/java/com/datatorrent/demos/machinedata/operator/CalculatorOperator.java*
and the *MachineInfoStreamCodec* in it.
Ram
On Tue, Apr 26, 2016 at 3:48 AM, Bhupesh Chawda
wrote:
> Hi
Ananth,
Could you make your pom.xml accessible somewhere ?
Ram
On Sat, Apr 16, 2016 at 3:30 PM, Ananth Gundabattula <
agundabatt...@gmail.com> wrote:
> Hello all,
>
> I would like to run a unit test code using the new Kafka Operator that
> supports the 0.9 version protocol.
>
> In this process,
(105)) - Exiting Application Master
>>>
>>> java.lang.NoSuchMethodError:
>>> org.apache.hadoop.conf.Configuration.addDeprecations([Lorg/apache/hadoop/conf/Configuration$DeprecationDelta;)V
>>>
>>> at
>>> org.apache.hadoop.yarn.conf.YarnConfi
34 AM
>>
>> To: "users@apex.incubator.apache.org"
>> Subject: Re: Error finding KafkaSinglePortStringInputOperator
>> (NoClassDefFoundError)
>>
>> Here is the sample from my working app,
>> https://github.com/sandeshh/streaming-benchmarks/bl
t; earliest
>
>
>
>
>
> dt.operator.MessageReader.consumer.prop.topic
>
> ad-server-log-topic
>
>
>
>
>
> dt.operator.MessageReader.consumer.prop.zookeeper
>
> ip-172-31-47-211.us-west-2.compute.internal:2181
>
>
Suhas,
Here is a bit more information on dealing with this class of errors, known
as
validation errors: Validation is a pre-deployment step to ensure that all
necessary constraints are met. It is discussed here:
http://docs.datatorrent.com/application_development/#validating-an-application
In thi
Jim,
There is some discussion of configuring memory here:
http://docs.datatorrent.com/troubleshooting/#configuration
http://docs.datatorrent.com/tutorials/topnwords-c2/#step-iv-customize-the-application-and-operators
http://docs.datatorrent.com/tutorials/topnwords-c7/
Ram
On Thu, Feb 25, 2016
e
wrote:
> Ram,
>
> The recovery path, when under the application directory, will be
> automatically copied to the new app directory when relaunch option is used.
> This is how the previous instance data is available to the new app.
>
> Thomas
>
> On Tue, Feb 16, 201
(because at some point some operator will forget and then we will
> reprocess a bunch of transactions; that would not be good!
>
>
>
> Jim
>
>
>
> *From:* Munagala Ramanath [mailto:r...@datatorrent.com]
> *Sent:* Tuesday, February 16, 2016 5:21 PM
> *To:* users@apex.
Why use the application id ? Could you generate and use a java.util.UUID
for example and save it in HDFS ?
Ram
On Tue, Feb 16, 2016 at 11:40 AM, Jim wrote:
> Good morning,
>
>
>
> I am new to Apex, Hadoop and Yarn (nothing like tackling something new, is
> there?).
>
>
>
> I have my first Apex
all the properties in the
> properties.xml, or just the high level ones from the system configuration
> that we want to change (i.e. will it still grab custom settings from the
> properties.xml if I use a config override)?
>
>
>
> Jim
>
>
>
> *From:* Munagala Ramanat
Jim,
On the sandbox, the start/stop scripts are at /sfw/hadoop/hadoop-2.2.0/sbin
so you can run stop-yarn.sh followed by start-yarn.sh but there may not be
any need to do so.
Your values are probably being overridden by values in ~/.dt/dt-site.xml
If you create config file called, say, foo.xml u
GW = GateWay
On Tue, Feb 2, 2016 at 10:37 AM, Sandesh Hegde
wrote:
> What is GW?
>
> On Tue, Feb 2, 2016 at 9:16 AM Pramod Immaneni
> wrote:
>
>> Good idea to handle it in GW.
>>
>> On Tue, Feb 2, 2016 at 8:50 AM, Thomas Weise
>> wrote:
>>
>>> Exactly, this doesn't make sense. I filed an enhan
Looking at apex-app-archetype/README.md in branch release-3.2 of
incubator-apex-core
and running the following maven command given there:
mvn archetype:generate -DarchetypeGroupId=org.apache.apex
-DarchetypeArtifactId=apex-app-archetype
-DarchetypeVersion=3.2.1-incubating-SNAPSHOT -DgroupId=com.ex
Since a single AbstractKafkaInputOperator has a single KafkaConsumer and
the latter has only a single topic,
it appears that the only option is to have one input operator per topic.
There also seems to be some machinery to get a reasonable mapping of input
operator partitions to
topic partitions,
I think a wizard will be more useful that just graphics, e.g.
Start with a question: What type of operator are you interested in ?
1. Input
2. Output
3. Compute
4. Unifier
Then refine the selection set from there and at some point down the
decision tree, print list of possible candidates.
You can
In OutputPortFieldAnnotation.java we have this:
public boolean optional() default true;
So looks like output ports are optional by default. If A annotates the port
as "optional = true", you're out of luck.
Ram
On Thu, Jan 21, 2016 at 6:57 AM, Yogi Devendra
wrote:
> I have Operator B extends
wrong but demo is not usually added as dependency;
> library & contrib is.
>
> ~ Chinmay
> On 12 Jan 2016 20:37, "Munagala Ramanath" wrote:
>
>> There are some concrete classes in the demos subdirectory of Malhar, see
>> for example:
>>
>>
>&
There are some concrete classes in the demos subdirectory of Malhar, see
for example:
*demos/frauddetect/src/main/java/com/datatorrent/demos/frauddetect/operator/HdfsStringOutputOperator.java*
*demos/wordcount/src/main/java/com/datatorrent/demos/wordcount/WordCountWriter.java*
Ram
On Tue, Jan 1
27 matches
Mail list logo
