Hi, I observed that adding json-schema-validator dependency in malhar-lib is leading to couple of dependency conflicts. Just to summarize, 1. If i go with version 2.0.1 of json-schema-validator, malhar does not build. There are some compilation errors and to fix those i had to add dependency of jsr. Malhar builds successfully with no compilation errors but DTFileTest.seekDTFile test fails. 2. I tried to go with latest version of json-schema-validator. With this i don't need to add dependency of jsr, malhar-iibrary builds successfully but JSON parser operator fails at runtime because of guava library conflicts.
Today i moved JSON parser in malhar-contrib and i did not face any issues during compilation / runtime. So i would be moving JSON parser to malhar-contrib. Thanks, Shubham On Thu, Jan 7, 2016 at 11:12 AM, Shubham Pathak <[email protected]> wrote: > Hi, > > JSON parser operator that is already existing in malhar-lib needs to be > enhanced. That enhancement needs dependency of json-schema-validator . So > we would need to add that dependency into malhar-lib. > > > Thanks, > Shubham > > On Thu, Jan 7, 2016 at 10:55 AM, Thomas Weise <[email protected]> > wrote: > >> Please address the dependency issue. >> >> On Wed, Jan 6, 2016 at 8:56 PM, Shubham Pathak <[email protected]> >> wrote: >> >> > Hi Chandni, >> > >> > This is enhancement to existing JSON Parser ( currently in malhar lib) >> to >> > take in schema for validations ( similar to XML / CSV parsers ). >> > >> > >> > Thanks, >> > Shubham >> > >> > On Thu, Jan 7, 2016 at 12:35 AM, Chandni Singh <[email protected] >> > >> > wrote: >> > >> > > Hey Shubham, >> > > >> > > Can the enhancement (extension) which adds a new dependency go into >> > > contrib? >> > > >> > > Thanks, >> > > Chandni >> > > >> > > On Wed, Jan 6, 2016 at 12:37 AM, Shubham Pathak < >> [email protected] >> > > >> > > wrote: >> > > >> > > > Chinmay, >> > > > >> > > > I tried to build malhar-library with the latest version of >> > > > json-schema-validator and it builds successfully without me having >> to >> > add >> > > > dependency for jsr. >> > > > >> > > > However, when i try to run the test, it throws a runtime error >> > > > [image: Inline image 1] >> > > > >> > > > >> > > > >> > > > Issue is that guava 11.0 is being considered instead of 16.01. >> > > > >> > > > One solution to this can be add <exclusion> for guava in apex common >> > but >> > > > that seems risky as it might lead to run time exceptions. >> > > > >> > > > For now I am going ahead with the approach i mentioned in my >> previous >> > > > mail. >> > > > >> > > > >> > > > Thomas, >> > > > No new operator is being added. Existing operator is being enhanced >> and >> > > > that requires a dependency to be added. >> > > > >> > > > >> > > > Thanks, >> > > > Shubham >> > > > >> > > > On Tue, Jan 5, 2016 at 11:25 PM, Thomas Weise < >> [email protected]> >> > > > wrote: >> > > > >> > > >> There was a discussion a while ago on why we don't want to add >> > > >> dependencies >> > > >> to malhar-library unless they are broadly applicable to many >> > operators. >> > > >> Please review that discussion. >> > > >> >> > > >> As for adding dependencies in general, please analyze the situation >> > > >> carefully before moving ahead.. You will want to look at licenses, >> > > >> possible >> > > >> conflicts with engine/Hadoop, transitive dependencies, functional >> > > overlap >> > > >> with existing dependencies etc. Also state why the dependency is >> > needed. >> > > >> >> > > >> Thomas >> > > >> >> > > >> >> > > >> On Tue, Jan 5, 2016 at 8:27 AM, Chinmay Kolhatkar < >> > > >> [email protected]> >> > > >> wrote: >> > > >> >> > > >> > Hey Shubham, >> > > >> > >> > > >> > FileSplitterInput was already present and devel-3 compiles fine >> > > without >> > > >> > jsr305 dependency (for Nullable). >> > > >> > It seems that adding json related dependency is affecting >> somewhere >> > > >> else. >> > > >> > >> > > >> > Can you please check if all the unit tests are passing after >> adding >> > > >> > findbug's jsr305 dependency? >> > > >> > >> > > >> > Also, There seems to be quite some versions of >> json-schema-validator >> > > >> after >> > > >> > 2.0.1, would it make sense to use latest version as dependency? >> > > >> > >> > > >> > Thanks, >> > > >> > Chinmay. >> > > >> > >> > > >> > >> > > >> > >> > > >> > ~ Chinmay. >> > > >> > >> > > >> > On Tue, Jan 5, 2016 at 5:29 PM, Shubham Pathak < >> > > [email protected] >> > > >> > >> > > >> > wrote: >> > > >> > >> > > >> > > Fixed by adding dependency of jsr . Now the pom looks like this >> > > >> > > >> > > >> > > <dependency> >> > > >> > > <groupId>com.github.fge</groupId> >> > > >> > > <artifactId>json-schema-validator</artifactId> >> > > >> > > <version>2.0.1</version> >> > > >> > > </dependency> >> > > >> > > <dependency> >> > > >> > > <groupId>com.google.code.findbugs</groupId> >> > > >> > > <artifactId>jsr305</artifactId> >> > > >> > > <version>1.3.9</version> >> > > >> > > <optional>true</optional> >> > > >> > > </dependency> >> > > >> > > >> > > >> > > Thanks, >> > > >> > > Shubham >> > > >> > > >> > > >> > > >> > > >> > > On Tue, Jan 5, 2016 at 2:35 PM, Shubham Pathak < >> > > >> [email protected]> >> > > >> > > wrote: >> > > >> > > >> > > >> > > > Hi, >> > > >> > > > >> > > >> > > > I am working on an operator that needs following dependency >> to >> > be >> > > >> added >> > > >> > > in >> > > >> > > > Malhar lib >> > > >> > > > <dependency> >> > > >> > > > <groupId>com.github.fge</groupId> >> > > >> > > > <artifactId>json-schema-validator</artifactId> >> > > >> > > > <version>2.0.1</version> >> > > >> > > > </dependency> >> > > >> > > > >> > > >> > > > But when i add this, and try to build malhar library, i get >> > > >> following >> > > >> > > error >> > > >> > > > >> > > >> > > >> > > >> > >> > > >> >> > > >> > >> /code/projects/Apache/incubator-apex-malhar/library/src/main/java/com/datatorrent/lib/io/fs/FileSplitterInput.java:[36,24] >> > > >> > > > cannot find symbol >> > > >> > > > [ERROR] symbol: class Nullable >> > > >> > > > [ERROR] location: package javax.annotation >> > > >> > > > >> > > >> > > > On further investigation i noticed, when i add the above >> > > dependency, >> > > >> > > > jsr305 jar doesn't show up in dependency hierarchy. >> > > >> > > > >> > > >> > > > If i remove the above dependency, jsr305 shows up. >> > > >> > > > >> > > >> > > > Can anyone please help ? >> > > >> > > > >> > > >> > > > Thanks, >> > > >> > > > Shubham >> > > >> > > > >> > > >> > > >> > > >> > >> > > >> >> > > > >> > > > >> > > >> > >> > >
