[jira] [Created] (AVRO-1926) lang/c++ build.sh doesn't work

[Simon Woodford (JIRA)]

Simon Woodford created AVRO-1926:


 Summary: lang/c++ build.sh  doesn't work
 Key: AVRO-1926
 URL: https://issues.apache.org/jira/browse/AVRO-1926
 Project: Avro
  Issue Type: Bug
  Components: c++
Affects Versions: 1.8.1
 Environment: Centos 7, SELinux, running from Docker container
Reporter: Simon Woodford
Priority: Minor
 Fix For: 1.8.3


The lang/c++/build.sh script makes frequent reference to the build directory, 
which is empty. Instead, make and all tests should be launched directly from 
lang/c++. 




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (AVRO-1925) Docker clash with SELinux

[Simon Woodford (JIRA)]

Simon Woodford created AVRO-1925:


 Summary: Docker clash with SELinux
 Key: AVRO-1925
 URL: https://issues.apache.org/jira/browse/AVRO-1925
 Project: Avro
  Issue Type: Bug
  Components: docker
Affects Versions: 1.8.1
 Environment: Linux - Centos 7 with SELinux running, avro found in 
/home/. 
Reporter: Simon Woodford
Priority: Minor
 Fix For: 1.8.3


When Docker is running in a labelling system like SELinux, it needs to be told 
to relabel any directories it wants to write to. 
In build.sh, we tell Docker that it can access /home/USER_NAME, /home/.m2 and 
/home/.gnupg, but we don't instruct it to relabel them. So if SELinux is 
running, once in the docker container, you cannot build anything as that 
involves writing. 
See e.g. the Volume Labels section in 
https://docs.docker.com/engine/tutorials/dockervolumes/ for more details. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (AVRO-1924) Variable named 'date' in IDL

[Niels Basjes (JIRA)]

Niels Basjes created AVRO-1924:
--

 Summary: Variable named 'date' in IDL
 Key: AVRO-1924
 URL: https://issues.apache.org/jira/browse/AVRO-1924
 Project: Avro
  Issue Type: Bug
Reporter: Niels Basjes
Assignee: Ryan Blue


I was compiling Apache Parquet and found that the switch from Avro 1.8.0 to 
1.8.1 broke their build.

The error: {code}
[ERROR] Failed to execute goal 
org.apache.avro:avro-maven-plugin:1.8.1:idl-protocol (schemas) ... 
org.apache.avro.compiler.idl.ParseException: Encountered " "date" "date "" at 
line 23, column 14.
[ERROR] Was expecting one of:
[ERROR]  ...
[ERROR] "@" ...
[ERROR] "`" ...
[ERROR] -> [Help 1]
{code}

As it turns out they have a test idl that contains this:

{code}
@namespace("org.apache.parquet.avro")
protocol Cars {
record Service {
long date;
}
}
{code}

And this change AVRO-1684 turned the word 'date' into something different for 
the idl compiler.

So changing the word 'date' into something else fixes the problem. 
Yet I think this is an undesirable effect for end user applications.

[~rdblue]: I assigned this to you implemented the mentioned change.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AVRO-1582) Json serialization of nullable fileds and fields with default values improvement.

[Zoltan Farkas (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15513391#comment-15513391
 ] 

Zoltan Farkas commented on AVRO-1582:
-

Hi Sean, I will provide a update from my side,  I am currently still stuck to 
get AVRO-1723 in.(working on Ryan's suggestions... he should get some code to 
review soon), after which I was planning to tackle this JIRA...

I will provide some detail on the implementation in case somebody wants to work 
on this.

My implementation is currently:
https://github.com/zolyfarkas/avro/blob/trunk/lang/java/avro/src/main/java/org/apache/avro/specific/ExtendedSpecificDatumWriter.java
https://github.com/zolyfarkas/avro/blob/trunk/lang/java/avro/src/main/java/org/apache/avro/reflect/ExtendedReflectDatumWriter.java
https://github.com/zolyfarkas/avro/blob/trunk/lang/java/avro/src/main/java/org/apache/avro/generic/ExtendedGenericDatumWriter.java
https://github.com/zolyfarkas/avro/blob/trunk/lang/java/avro/src/main/java/org/apache/avro/io/ExtendedJsonDecoder.java
https://github.com/zolyfarkas/avro/blob/trunk/lang/java/avro/src/main/java/org/apache/avro/io/ExtendedJsonEncoder.java

here is what needs to be considered:

1) Currently implementation does: a) optimizes union {null, something} b) 
omits/infers fields that are equal with the default values. b) is very useful 
in the world that uses schemas by reducing the size of the payload. But I can 
see issues with the schema-less crowd, where they need the fields because they 
don't have the schema... which is why some people suggested separating a) from 
b)
2) I still need to move over unit tests that I have outside of the library.
3) there is more potential for improvement here, for example: union {null, int, 
string}, union {double, record}... can also be jsonized better, which I have on 
my todo list, and will be in my implementation sometime in the next 6 months... 
this might change the approach the current implementation takes...

Unfortunately my time available for this is limited... and since our use cases 
are covered in the fork we use, this is currently low priority in my list...

> Json serialization of nullable fileds and fields with default values 
> improvement.
> -
>
> Key: AVRO-1582
> URL: https://issues.apache.org/jira/browse/AVRO-1582
> Project: Avro
>  Issue Type: Improvement
>  Components: java
>Affects Versions: 1.8.0
>Reporter: Zoltan Farkas
> Attachments: AVRO-1582-PATCH
>
>
> Currently serializing a nullable field of type union like:
> "type" : ["null","some type"]
> when serialized as JSON results in:  
> "field":{"some type":"value"}
> when it could be:
> "field":"value"
> Also fields that equal the the default value can be omitted from the 
> serialized data. This is possible because the reader will have the writer's 
> schema and can infer the field values. This reduces the size of the json 
> messages.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AVRO-1923) Recursive record causes StackOverflow in GenericData.toString

[Niels Basjes (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AVRO-1923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Niels Basjes updated AVRO-1923:
---
Status: Patch Available  (was: Open)

> Recursive record causes StackOverflow in GenericData.toString
> -
>
> Key: AVRO-1923
> URL: https://issues.apache.org/jira/browse/AVRO-1923
> Project: Avro
>  Issue Type: Bug
>Reporter: Niels Basjes
>Assignee: Niels Basjes
> Attachments: AVRO-1923-Unittest.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AVRO-1923) Recursive record causes StackOverflow in GenericData.toString

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15513224#comment-15513224
 ] 

ASF GitHub Bot commented on AVRO-1923:
--

GitHub user nielsbasjes opened a pull request:

https://github.com/apache/avro/pull/128

AVRO-1923: Stop infinite recursion in GenericData.toString

Simply keep count of the recursion depth and stop at 50 (which seems to me 
like a good value)

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/nielsbasjes/avro AVRO-1923

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/avro/pull/128.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #128


commit 1a0c0ee22e891537e58f713d3e5dee11b3396da7
Author: Niels Basjes 
Date:   2016-09-22T12:42:47Z

AVRO-1923: Stop infinite recursion in GenericData.toString




> Recursive record causes StackOverflow in GenericData.toString
> -
>
> Key: AVRO-1923
> URL: https://issues.apache.org/jira/browse/AVRO-1923
> Project: Avro
>  Issue Type: Bug
>Reporter: Niels Basjes
>Assignee: Niels Basjes
> Attachments: AVRO-1923-Unittest.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] avro pull request #128: AVRO-1923: Stop infinite recursion in GenericData.to...

[nielsbasjes]

GitHub user nielsbasjes opened a pull request:

https://github.com/apache/avro/pull/128

AVRO-1923: Stop infinite recursion in GenericData.toString

Simply keep count of the recursion depth and stop at 50 (which seems to me 
like a good value)

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/nielsbasjes/avro AVRO-1923

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/avro/pull/128.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #128


commit 1a0c0ee22e891537e58f713d3e5dee11b3396da7
Author: Niels Basjes 
Date:   2016-09-22T12:42:47Z

AVRO-1923: Stop infinite recursion in GenericData.toString




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Assigned] (AVRO-1923) Recursive record causes StackOverflow in GenericData.toString

[Niels Basjes (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AVRO-1923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Niels Basjes reassigned AVRO-1923:
--

Assignee: Niels Basjes

> Recursive record causes StackOverflow in GenericData.toString
> -
>
> Key: AVRO-1923
> URL: https://issues.apache.org/jira/browse/AVRO-1923
> Project: Avro
>  Issue Type: Bug
>Reporter: Niels Basjes
>Assignee: Niels Basjes
> Attachments: AVRO-1923-Unittest.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AVRO-1923) Recursive record causes StackOverflow in GenericData.toString

[Niels Basjes (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AVRO-1923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Niels Basjes updated AVRO-1923:
---
Attachment: AVRO-1923-Unittest.patch

A unit test that reproduces the problem.

> Recursive record causes StackOverflow in GenericData.toString
> -
>
> Key: AVRO-1923
> URL: https://issues.apache.org/jira/browse/AVRO-1923
> Project: Avro
>  Issue Type: Bug
>Reporter: Niels Basjes
> Attachments: AVRO-1923-Unittest.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (AVRO-1923) Recursive record causes StackOverflow in GenericData.toString

[Niels Basjes (JIRA)]

Niels Basjes created AVRO-1923:
--

 Summary: Recursive record causes StackOverflow in 
GenericData.toString
 Key: AVRO-1923
 URL: https://issues.apache.org/jira/browse/AVRO-1923
 Project: Avro
  Issue Type: Bug
Reporter: Niels Basjes






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)