Hi Scott,
I’ll have time to look in to this tomorrow for you. I’ve also sent this to the dev@heron list in case someone else would be interested. - Josh On Fri, Jun 4, 2021 at 2:30 PM Scott Mayne <[email protected]> wrote: > Hi all, > > I asked a question about running ECO in local mode a while back, thanks to > Josh for suggesting the verbose flag but nothing was added to the console > that shed any light on why it exited right away. > > In the meantime, I got things running in IntelliJ > using org.apache.heron.simulator.Simulator, after creating the topology > using org.apache.heron.api.topology.TopologyBuilder. > > Now that that's working, we've circled back to the ECO/IDE/YAML question. > We have a bunch of Storm 1.0.6 ETLs that we run using command line Flux for > cluster deploys, and in an IDE with a main() method using Flux as well (see > bottom of message for my unsuccessful attempt to mimic the Flux way with > Eco). > > Obviously Storm 1.0.6 is getting a little long in the tooth and we're > exploring moving to Heron (and liking what we see so far). > > A nice thing about the Storm setup was we use one YAML file to define the > topologies in both scenarios. So far in Heron, it would be ECO/YAML for > cluster (not that we've gotten that far yet) and TopologyBuilder in the > IDE for development. > > Would be nice to not have this difference and use YAML for both (avoid > mismatches between the two, etc.), so my question is: Has anyone set up > Heron in IntelliJ (or other IDE) so it can read a (strictly Heron) topology > from a YAML file and run, hit breakpoints, etc. (with ECO or otherwise)? If > so, how? > > As background, these Storm ETLs were written before I joined the team, and > I'm relatively new to streaming frameworks, so I might be missing something > obvious here. > > For example, I just removed the Storm artifacts from the POM, added Heron, > and modified the ETL code as needed. Haven't installed Heron binaries or > anything like that on my laptop, not sure if that's a prerequisite. > > Thanks, > Scott > > public class KafkaTopology { > > public static void main(String[] args) throws Exception { > > List<String> argList = new ArrayList<String>(Arrays.asList(args)); > String file = > KafkaTopology.class.getClassLoader().getResource("topology.yaml").getFile(); > argList.add("local"); > argList.add("--eco-config-file"); > argList.add(file); > > file = > KafkaTopology.class.getClassLoader().getResource("dev.properties").getFile(); > argList.add("--props"); > argList.add(file); > > argList.add("--sleep"); // something from Flux to make it run for a > while > argList.add("36000000"); > > String[] ecoArgs = argList.toArray(new String[argList.size()]); > > Eco.main(ecoArgs); > > } > } > > > -- Sent from A Mobile Device
