just to clarify, adding:
serializers.registry.string.class=org.apache.samza.
serializers.StringSerdeFactory
systems.kafka.streams.myTopic.samza.msg.serde=string
to the property file and updating the java source to:
System.out.println((String)envelope.getMessage());
Did the trick. I've updated th
Huzzah! I ... have ... text showing!
This has been enough of a trial that I think I'll convert this into a very
simple sample project for the repo, if you guys are interested.
Diff coming once I have it cleaned up into something less ugly.
-Ash
On Mon, Mar 23, 2015 at 9:27 PM, Chinmay Soman
w
>I changed the systems.kafka.samza.msg.serde=json to 'string' a while back,
but that caused a separate exception. However that was many, MANY attempts
ago.
This may not work because that will set all serialization formats (input
and output) to json / string. In your case you're inputting string a
There's also another bug I just found in my code. But I'll try that as
well (and post when I have an updated diff)
On Mon, Mar 23, 2015 at 9:24 PM, Chinmay Soman
wrote:
> Since you're producing String data to 'myTopic', can you try setting the
> string serialization in your config ?
>
>
> seria
Since you're producing String data to 'myTopic', can you try setting the
string serialization in your config ?
serializers.registry.string.class=org.apache.samza.serializers.StringSerdeFactory
systems.kafka.streams.myTopic.samza.msg.serde=string
On Mon, Mar 23, 2015 at 9:17 PM, Ash W Matheson
more info - new exception message:
Exception in thread "main"
org.apache.samza.system.SystemConsumersException: Cannot deserialize an
incoming message.
Updated the diff in pastebin with the changes.
On Mon, Mar 23, 2015 at 8:41 PM, Ash W Matheson
wrote:
> Gah! Yeah, those were gone several re
Gah! Yeah, those were gone several revisions ago but didn't get nuked in
the last iteration.
OK, let me do a quick test to see if that was my problem all along.
On Mon, Mar 23, 2015 at 8:38 PM, Navina Ramesh wrote:
> Hey Ash,
> I was referring to the lines before the try block.
>
> Map jsonObj
Hey Ash,
I was referring to the lines before the try block.
Map jsonObject = (Map)
envelope.getMessage();
WikipediaFeedEvent event = new WikipediaFeedEvent(jsonObject);
try {
System.out.println("[DWH] should see this");
System.out.println(event.getRawEvent());
…
Did you rem
Just looking at the diff I posted and it's:
1. try {
2. - Map parsedJsonObject = parse(event.getRawEvent(
));
3. + System.out.println("[DWH] should see this");
4. + System.out.println(event.getRawEvent());
5. + // Map parsedJsonObject = parse(
event.g
I'm in transit right now but if memory serves me everything should be
commented out of that method except for the System.out.println call. I'll
be home shortly and can confirm.
On Mar 23, 2015 7:28 PM, "Navina Ramesh"
wrote:
> Hi Ash,
> I just ran wikipedia-parser with your patch. Looks like you
Hi all,
I think it is good idea to improve our presence in stackoverflow.com because:
* any new user/developer mostly googles/searches stackoverflow for resolving
issues before hitting us on the mailing list; This requires the user to
subscribe to the mailing list -> unnecessary overhead, apart
Hi Ash,
I just ran wikipedia-parser with your patch. Looks like you have set the
message serde correctly in the configs. However, the original code still
converts it into a Map for consumption in the WikipediaFeedEvent.
I am seeing the following (expected):
2015-03-23 19:17:49 SamzaContainerExcept
If anyone's interested, I've posted a diff of the project here:
http://pastebin.com/6ZW6Y1Vu
and the python publisher here: http://pastebin.com/2NvTFDFx
if you want to take a stab at it.
On Mon, Mar 23, 2015 at 6:04 PM, Ash W Matheson
wrote:
> Ok, so very simple test, all running on a local mac
Ok, so very simple test, all running on a local machine, not across
networks and all in the hello-samza repo this time around.
I've got the datapusher.py file set up to push data into localhost. One
event per second.
And a modified hello-samza where I've modified the
WikipediaParserStreamTask.java
Ok, got the console up and running, but what should I be looking for?
On Mon, Mar 23, 2015 at 5:32 PM, Chinmay Soman
wrote:
> You can use Jconsole / Jvisualvm and connect to the JMX port as documented
> here:
>
> http://samza.apache.org/learn/documentation/0.8/container/jmx.html
>
>
>
> On Mon,
You can use Jconsole / Jvisualvm and connect to the JMX port as documented
here:
http://samza.apache.org/learn/documentation/0.8/container/jmx.html
On Mon, Mar 23, 2015 at 5:23 PM, Ash W Matheson
wrote:
> Ahh, I was going to add it to the run-class.sh script.
>
> Yeah, it's already there by d
Ahh, I was going to add it to the run-class.sh script.
Yeah, it's already there by default:
# Metrics
metrics.reporters=snapshot,jmx
metrics.reporter.snapshot.class=org.apache.samza.metrics.reporter.MetricsSnapshotReporterFactory
metrics.reporter.snapshot.stream=kafka.metrics
metrics.reporter.jm
read: I'm a C++ programmer looking at Java for the first time in > 10 years
On Mon, Mar 23, 2015 at 5:13 PM, Ash W Matheson
wrote:
> I'm assuming I have Jmx defined ... where would that get set?
>
> On Mon, Mar 23, 2015 at 5:08 PM, Chinmay Soman
> wrote:
>
>> Hey Ash,
>>
>> Can you see your job
Oh, I just meant in your job config:
metrics.reporter.jmx.class=org.apache.samza.metrics.reporter.JmxReporterFactory
metrics.reporters=jmx
On Mon, Mar 23, 2015 at 5:13 PM, Ash W Matheson
wrote:
> I'm assuming I have Jmx defined ... where would that get set?
>
> On Mon, Mar 23, 2015 at 5:08 PM,
I'm assuming I have Jmx defined ... where would that get set?
On Mon, Mar 23, 2015 at 5:08 PM, Chinmay Soman
wrote:
> Hey Ash,
>
> Can you see your job metrics (if you have the Jmx metrics defined) to see
> if your job is actually doing anything ? My only guess at this point is the
> process met
Hey Ash,
Can you see your job metrics (if you have the Jmx metrics defined) to see
if your job is actually doing anything ? My only guess at this point is the
process method is not being called because somehow there's no incoming
data. I could be totally wrong of course.
On Mon, Mar 23, 2015 at 4
> On March 23, 2015, 11:29 p.m., Chris Riccomini wrote:
> > * Nit: 2 space, not 4 space for indentation.
> > * It's kind of odd to have exceptionHandler.maybeHandle {
> > maybeHandle(coordinator, envelope, tryBlock = { ... I had a comment in the
> > RB about consolidating this.
> > * TestTaskIn
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32407/#review77470
---
* Nit: 2 space, not 4 space for indentation.
* It's kind of odd to h
Just to be clear, here's what's changed from the default hello-samza repo:
wikipedia-parser.properties==
task.inputs=kafka.myTopic
systems.kafka.consumer.zookeeper.connect=
ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com:2181/
systems.kafka.consumer.auto.offset.reset=smallest
yep, modified log4j.xml to look like this:
Not sure what you mean by #2.
However, I'm running now, not seeing any exceptions, but still not seeing
any output from System.out.println(...)
On Mon, Mar 23, 2015 at 11:29 AM, Naveen Somasundaram <
nsomasunda...@linkedin.com.inva
I would like to decouple Samza properties with the custom ones (if
possible).
- Shekar
On Mon, Mar 23, 2015 at 3:12 PM, Shekar Tippur wrote:
> Hello,
>
> I have a custom properties file with name value pairs.
>
> name1 value1
> name2 value2
> name3 value3
>
> I want to pass this to Yarn job. Wh
Hello,
I have a custom properties file with name value pairs.
name1 value1
name2 value2
name3 value3
I want to pass this to Yarn job. What is the best way to achieve this? This
works well when we declare locally or thread job.
- Shekar
Hey Ash,
1. Did you happen to modify your log4j.xml ?
2. Can you print the class path that was printed when the job
started ? I am wondering if log4j was not loaded or not present in the path
where it’s looking for. If you have been using hello samza, it should have
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32407/
---
Review request for samza, Yan Fang, Chinmay Soman, Chris Riccomini, Navina
Rames
Hey Jordi,
This is because you're sending String and not json in your output topic.
Try setting string on the output stream as well (if you haven't already).
If you have done that - then please enable debug mode and attach the log
somewhere so that we can take a look.
On Mon, Mar 23, 2015 at 9:5
Looks like that was one error.
I have set the property like this:
systems.kafka.streams.syslog.samza.msg.serde=string
But I am still getting the same error. Now I am seeing a different thing in the
log previous to the exception:
23 mar 2015 05:49:31 INFO KafkaSystemProducer - Creating
Yeah, to make sure that I could get at the topic on the server I re-ran the
producer and kafka-consumer to ensure I hadn't done something stupid - can
produce and consume data at will. So I'm nuking the 'hello-samza' folder
and restarting from scratch, just to ensure I haven't messed something up
Have you tried setting this :
systems.kafka.streams.syslog.samza.msg.serde=string // And assuming
you've defined a 'string' serializer in your config
OR
systems.kafka.streams.syslog.samza.msg.serde=json // Depending on the
corresponding format of your input data
On Mon, Mar 23, 2015 at 9:
Hey Ash,
Yeah - I've confirmed that System.out.println should result in a message
getting logged in the 'stdout' (at least while running in the local YARN
mode).
Not sure why you're not seeing the same behaviour. I'm assuming you've
already confirmed that you're seeing data in your input Kafka to
Hi,
As I understand it, I am setting "kafka" as the system name, "beste" as the
output topic in the system and "syslog" as the input topic. Both topics syslog
and beste are working correctly as I am streaming some syslogs to the "syslog"
topic and I am testing "beste" with an internal applicati
Hey Jordi,
I see 3 different stream names.
1. new SystemStream("kafka", "beste");
2. task.inputs=kafka.syslog
3. systems.kafka.streams.frogain.samza.msg.serde=json
Just for a sanity check, can you double check you're setting the config
params for the correct stream ?
On Mon, Mar 23, 2015 at
Hello,
I have managed to get samza up and running an simple test job that just sends
the received message. This is the code:
public class job1 implements StreamTask {
private final SystemStream OUTPUT_STREAM = new SystemStream("kafka",
"beste");
public void process(IncomingMess
37 matches
Mail list logo
