date:20190822

[jira] [Created] (AVRO-2519) Use NettyTransceiver repeatedly connect to remote server which doesn't support current RPC or unavailable will cause " java.lang.OutOfMemoryError: Direct buffer memory"

2019-08-22 Thread Xun Wang (Jira)

Xun Wang created AVRO-2519:
--

 Summary: Use NettyTransceiver repeatedly connect to remote server 
which doesn't support current RPC or unavailable will cause " 
java.lang.OutOfMemoryError: Direct buffer memory"
 Key: AVRO-2519
 URL: https://issues.apache.org/jira/browse/AVRO-2519
 Project: Apache Avro
  Issue Type: Bug
  Components: java
Affects Versions: 1.9.0
 Environment: java version "12.0.1" 2019-04-16

Java(TM) SE Runtime Environment (build 12.0.1+12)

Java HotSpot(TM) 64-Bit Server VM (build 12.0.1+12, mixed mode, sharing)
Reporter: Xun Wang
 Attachments: avro-rpc-quickstart test.zip

I just modify the code from 
https://github.com/phunt/avro-rpc-quickstart。Execute the modified code can 100% 
reproduce this issue. I have attached the test project file. When use 
NettyTransceiver repeatedly connect to remote server which doesn't support 
current RPC or unavailable will cause " java.lang.OutOfMemoryError: Direct 
buffer memory"

The source to execute:

/**

 * Licensed to the Apache Software Foundation (ASF) under one

 * or more contributor license agreements.  See the NOTICE file

 * distributed with this work for additional information

 * regarding copyright ownership.  The ASF licenses this file

 * to you under the Apache License, Version 2.0 (the

 * "License"); you may not use this file except in compliance

 * with the License.  You may obtain a copy of the License at

 *

 *     http://www.apache.org/licenses/LICENSE-2.0

 *

 * Unless required by applicable law or agreed to in writing, software

 * distributed under the License is distributed on an "AS IS" BASIS,

 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

 * See the License for the specific language governing permissions and

 * limitations under the License.

 */

 

package example;

 

import java.io.IOException;

import java.net.InetAddress;

import java.net.InetSocketAddress;

 

import org.apache.avro.ipc.netty.NettyServer;

import org.apache.avro.ipc.netty.NettyTransceiver;

import org.apache.avro.ipc.Server;

import org.apache.avro.ipc.specific.SpecificRequestor;

import org.apache.avro.ipc.specific.SpecificResponder;

import org.apache.avro.util.Utf8;

 

import example.proto.Mail;

import example.proto.Message;

 

/**

 * Start a server, attach a client, and send a message.

 */

public class Main {

public static class MailImpl implements Mail {

// in this simple example just return details of the message

public Utf8 send(Message message) {

System.out.println("Sending message");

return new Utf8("Sending message to " + message.getTo().toString() + " from " + 
message.getFrom().toString()

+ " with body " + message.getBody().toString());

}

}

 

private static Server server;

 

private static void startServer() throws IOException {

server = new NettyServer(new SpecificResponder(Mail.class, new MailImpl()), new 
InetSocketAddress(65111));

// the server implements the Mail protocol (MailImpl)

}

 

public static void main(String[] args) throws IOException {

//        if (args.length != 3) {

//            System.out.println("Usage:   ");

//            System.exit(1);

//        }

 

System.out.println("Starting server");

// usually this would be another app, but for simplicity

//        startServer();

System.out.println("Server started");

 

while (true) {

NettyTransceiver client = null;

try {

client = new NettyTransceiver(new 
InetSocketAddress(InetAddress.getByName("129.28.206.27"), 7999), 3000l);

// client code - attach to the server and send a message

Mail proxy = (Mail) SpecificRequestor.getClient(Mail.class, client);

System.out.println("Client built, got proxy");

 

// fill in the Message record and send it

Message message = new Message();

message.setTo(new Utf8(args[0]));

message.setFrom(new Utf8(args[1]));

message.setBody(new Utf8(args[2]));

System.out.println("Calling proxy.send with message:  " + message.toString());

System.out.println("Result: " + proxy.send(message));

} catch (Exception e) {

System.out.println(e.getMessage());

} finally {

// cleanup

if(client != null) {

client.close();

}

}

 

}

 

//        server.close();

}

}

The error log:

Starting server

Server started

Error connecting to /129.28.206.27:7999

Error connecting to /129.28.206.27:7999

Error connecting to /129.28.206.27:7999

Error connecting to /129.28.206.27:7999

Error connecting to /129.28.206.27:7999

Error connecting to /129.28.206.27:7999

Error connecting to /129.28.206.27:7999

Exception in thread "main" java.lang.OutOfMemoryError: Direct buffer memory

at java.base/java.nio.Bits.reserveMemory(Bits.java:175)

at java.base/java.nio.DirectByteBuffer.(DirectByteBuffer.java:118)

at java.base/java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:317)

at

[jira] [Updated] (AVRO-2518) Ruby Avro doesn't validate enum fields with default values

2019-08-22 Thread Kyle Boyer (Jira)



 [ 
https://issues.apache.org/jira/browse/AVRO-2518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kyle Boyer updated AVRO-2518:
-
Attachment: 0001-AVRO-2518-Add-enum-field-with-default-value-test.patch
0002-AVRO-2518-Validate-enum-field-with-default-values.patch
Status: Patch Available  (was: Open)

> Ruby Avro doesn't validate enum fields with default values
> --
>
> Key: AVRO-2518
> URL: https://issues.apache.org/jira/browse/AVRO-2518
> Project: Apache Avro
>  Issue Type: Bug
>  Components: ruby
>Affects Versions: 1.9.0
> Environment: Centos 7 Docker - ruby 2.5.5p157 (2019-03-15 revision 
> 67260) [x86_64-linux-musl]
> Mac OS X 10.14.X - ruby 2.3.7p456 (2018-03-28 revision 63024) 
> [universal.x86_64-darwin18]
>Reporter: Kyle Boyer
>Priority: Major
>  Labels: easyfix, pull-request-available, test
> Attachments: 
> 0001-AVRO-2518-Add-enum-field-with-default-value-test.patch, 
> 0002-AVRO-2518-Validate-enum-field-with-default-values.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> As of Avro 1.9, the 
> [specification|https://avro.apache.org/docs/1.9.0/spec.html#Enums] states 
> that Enums can have a default value. With the following schema(or similar 
> schemas having enum fields with default values):
> {code:json}
> {
>   "type": "record",
>   "name": "enum_field_default",
>   "fields": [
> {
>   "name": "msg",
>   "type": "string"
> },
> {
>   "name": "logClass",
>   "type": {
> "type": "enum",
> "name": "logClass",
> "symbols": [
>   "UNCATEGORIZED",
>   "E1",
>   "E2",
>   "E3",
>   "E4",
>   "E5",
>   "E6",
>   "E7",
>   "E8",
>   "E9",
>   "E10"
> ]
>   },
>   "default": "UNCATEGORIZED"
> }
>   ]
> }
> {code} 
> We are unable to validate this message using Ruby Avro 1.9, however it works 
> fine using Java Avro 1.9: 
> {code:json}
> {"msg": "boom"}
> {code}
> A PR has been created to fix this issue: 
> https://github.com/apache/avro/pull/618



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Updated] (AVRO-2518) Ruby Avro doesn't validate enum fields with default values

2019-08-22 Thread Kyle Boyer (Jira)



 [ 
https://issues.apache.org/jira/browse/AVRO-2518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kyle Boyer updated AVRO-2518:
-
Description: 
As of Avro 1.9, the 
[specification|https://avro.apache.org/docs/1.9.0/spec.html#Enums] states that 
Enums can have a default value. With the following schema(or similar schemas 
having enum fields with default values):
{code:json}
{
  "type": "record",
  "name": "enum_field_default",
  "fields": [
{
  "name": "msg",
  "type": "string"
},
{
  "name": "logClass",
  "type": {
"type": "enum",
"name": "logClass",
"symbols": [
  "UNCATEGORIZED",
  "E1",
  "E2",
  "E3",
  "E4",
  "E5",
  "E6",
  "E7",
  "E8",
  "E9",
  "E10"
]
  },
  "default": "UNCATEGORIZED"
}
  ]
}
{code} 

We are unable to validate this message using Ruby Avro 1.9, however it works 
fine using Java Avro 1.9: 
{code:json}
{"msg": "boom"}
{code}

A PR has been created to fix this issue: https://github.com/apache/avro/pull/618

  was:
As of Avro 1.9, the 
[specification|https://avro.apache.org/docs/1.9.0/spec.html#Enums] states that 
Enums can have a default value. With the following schema(or similar schemas 
having enum fields with default values):
{code:json}
{
  "type": "record",
  "name": "enum_field_default",
  "fields": [
{
  "name": "msg",
  "type": "string"
},
{
  "name": "logClass",
  "type": {
"type": "enum",
"name": "logClass",
"symbols": [
  "UNCATEGORIZED",
  "E1",
  "E2",
  "E3",
  "E4",
  "E5",
  "E6",
  "E7",
  "E8",
  "E9",
  "E10"
]
  },
  "default": "UNCATEGORIZED"
}
  ]
}
{code} 

We are unable to validate this message using Ruby Avro 1.9, however it works 
fine using Java Avro 1.9: 
{code:json}
{"msg": "boom"}
{code}

A draft PR has been created to fix this issue: 
https://github.com/apache/avro/pull/618


> Ruby Avro doesn't validate enum fields with default values
> --
>
> Key: AVRO-2518
> URL: https://issues.apache.org/jira/browse/AVRO-2518
> Project: Apache Avro
>  Issue Type: Bug
>  Components: ruby
>Affects Versions: 1.9.0
> Environment: Centos 7 Docker - ruby 2.5.5p157 (2019-03-15 revision 
> 67260) [x86_64-linux-musl]
> Mac OS X 10.14.X - ruby 2.3.7p456 (2018-03-28 revision 63024) 
> [universal.x86_64-darwin18]
>Reporter: Kyle Boyer
>Priority: Major
>  Labels: easyfix, pull-request-available, test
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> As of Avro 1.9, the 
> [specification|https://avro.apache.org/docs/1.9.0/spec.html#Enums] states 
> that Enums can have a default value. With the following schema(or similar 
> schemas having enum fields with default values):
> {code:json}
> {
>   "type": "record",
>   "name": "enum_field_default",
>   "fields": [
> {
>   "name": "msg",
>   "type": "string"
> },
> {
>   "name": "logClass",
>   "type": {
> "type": "enum",
> "name": "logClass",
> "symbols": [
>   "UNCATEGORIZED",
>   "E1",
>   "E2",
>   "E3",
>   "E4",
>   "E5",
>   "E6",
>   "E7",
>   "E8",
>   "E9",
>   "E10"
> ]
>   },
>   "default": "UNCATEGORIZED"
> }
>   ]
> }
> {code} 
> We are unable to validate this message using Ruby Avro 1.9, however it works 
> fine using Java Avro 1.9: 
> {code:json}
> {"msg": "boom"}
> {code}
> A PR has been created to fix this issue: 
> https://github.com/apache/avro/pull/618



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Updated] (AVRO-2518) Ruby Avro doesn't validate enum fields with default values

2019-08-22 Thread Kyle Boyer (Jira)



 [ 
https://issues.apache.org/jira/browse/AVRO-2518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kyle Boyer updated AVRO-2518:
-
Description: 
As of Avro 1.9, the 
[specification|https://avro.apache.org/docs/1.9.0/spec.html#Enums] states that 
Enums can have a default value. With the following schema(or similar schemas 
having enum fields with default values):
{code:json}
{
  "type": "record",
  "name": "enum_field_default",
  "fields": [
{
  "name": "msg",
  "type": "string"
},
{
  "name": "logClass",
  "type": {
"type": "enum",
"name": "logClass",
"symbols": [
  "UNCATEGORIZED",
  "E1",
  "E2",
  "E3",
  "E4",
  "E5",
  "E6",
  "E7",
  "E8",
  "E9",
  "E10"
]
  },
  "default": "UNCATEGORIZED"
}
  ]
}
{code} 

We are unable to validate this message using Ruby Avro 1.9, however it works 
fine using Java Avro 1.9: 
{code:json}
{"msg": "boom"}
{code}

A draft PR has been created to fix this issue: 
https://github.com/apache/avro/pull/618

  was:
As of Avro 1.9, the 
[specification|https://avro.apache.org/docs/1.9.0/spec.html#Enums] states that 
Enums can have a default value. With the following schema(or similar schemas 
having enum fields with default values):
{code:json}
{
  "type": "record",
  "name": "enum_field_default",
  "fields": [
{
  "name": "msg",
  "type": "string"
},
{
  "name": "logClass",
  "type": {
"type": "enum",
"name": "logClass",
"symbols": [
  "UNCATEGORIZED",
  "E1",
  "E2",
  "E3",
  "E4",
  "E5",
  "E6",
  "E7",
  "E8",
  "E9",
  "E10"
]
  },
  "default": "UNCATEGORIZED"
}
  ]
}
{code} 

We are unable to validate this message using Ruby Avro 1.9, however it works 
fine using Java Avro 1.9: 
{code:json}
{"msg": "boom"}
{code}

A draft PR has been created to fix this issue: 
https://github.com/apache/avro/pull/614


> Ruby Avro doesn't validate enum fields with default values
> --
>
> Key: AVRO-2518
> URL: https://issues.apache.org/jira/browse/AVRO-2518
> Project: Apache Avro
>  Issue Type: Bug
>  Components: ruby
>Affects Versions: 1.9.0
> Environment: Centos 7 Docker - ruby 2.5.5p157 (2019-03-15 revision 
> 67260) [x86_64-linux-musl]
> Mac OS X 10.14.X - ruby 2.3.7p456 (2018-03-28 revision 63024) 
> [universal.x86_64-darwin18]
>Reporter: Kyle Boyer
>Priority: Major
>  Labels: easyfix, pull-request-available, test
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> As of Avro 1.9, the 
> [specification|https://avro.apache.org/docs/1.9.0/spec.html#Enums] states 
> that Enums can have a default value. With the following schema(or similar 
> schemas having enum fields with default values):
> {code:json}
> {
>   "type": "record",
>   "name": "enum_field_default",
>   "fields": [
> {
>   "name": "msg",
>   "type": "string"
> },
> {
>   "name": "logClass",
>   "type": {
> "type": "enum",
> "name": "logClass",
> "symbols": [
>   "UNCATEGORIZED",
>   "E1",
>   "E2",
>   "E3",
>   "E4",
>   "E5",
>   "E6",
>   "E7",
>   "E8",
>   "E9",
>   "E10"
> ]
>   },
>   "default": "UNCATEGORIZED"
> }
>   ]
> }
> {code} 
> We are unable to validate this message using Ruby Avro 1.9, however it works 
> fine using Java Avro 1.9: 
> {code:json}
> {"msg": "boom"}
> {code}
> A draft PR has been created to fix this issue: 
> https://github.com/apache/avro/pull/618



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Created] (AVRO-2518) Ruby Avro doesn't validate enum fields with default values

2019-08-22 Thread Kyle Boyer (Jira)

Kyle Boyer created AVRO-2518:


 Summary: Ruby Avro doesn't validate enum fields with default values
 Key: AVRO-2518
 URL: https://issues.apache.org/jira/browse/AVRO-2518
 Project: Apache Avro
  Issue Type: Bug
  Components: ruby
Affects Versions: 1.9.0
 Environment: Centos 7 Docker - ruby 2.5.5p157 (2019-03-15 revision 
67260) [x86_64-linux-musl]

Mac OS X 10.14.X - ruby 2.3.7p456 (2018-03-28 revision 63024) 
[universal.x86_64-darwin18]
Reporter: Kyle Boyer


As of Avro 1.9, the 
[specification|https://avro.apache.org/docs/1.9.0/spec.html#Enums] states that 
Enums can have a default value. With the following schema(or similar schemas 
having enum fields with default values):
{code:json}
{
  "type": "record",
  "name": "enum_field_default",
  "fields": [
{
  "name": "msg",
  "type": "string"
},
{
  "name": "logClass",
  "type": {
"type": "enum",
"name": "logClass",
"symbols": [
  "UNCATEGORIZED",
  "E1",
  "E2",
  "E3",
  "E4",
  "E5",
  "E6",
  "E7",
  "E8",
  "E9",
  "E10"
]
  },
  "default": "UNCATEGORIZED"
}
  ]
}
{code} 

We are unable to validate this message using Ruby Avro 1.9, however it works 
fine using Java Avro 1.9: 
{code:json}
{"msg": "boom"}
{code}

A draft PR has been created to fix this issue: 
https://github.com/apache/avro/pull/614



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Commented] (AVRO-2515) Enable direct access to utility methods in ProtobufData

2019-08-22 Thread Hudson (Jira)



[ 
https://issues.apache.org/jira/browse/AVRO-2515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913272#comment-16913272
 ] 

Hudson commented on AVRO-2515:
--

SUCCESS: Integrated in Jenkins build AvroJava #727 (See 
[https://builds.apache.org/job/AvroJava/727/])
AVRO-2515: Expand access of ProtobufData methods (#612) (fokko: 
[https://github.com/apache/avro/commit/bce00d9c4a44f66c6906e66cde21938935c00fc5])
* (edit) 
lang/java/protobuf/src/main/java/org/apache/avro/protobuf/ProtobufData.java


> Enable direct access to utility methods in ProtobufData
> ---
>
> Key: AVRO-2515
> URL: https://issues.apache.org/jira/browse/AVRO-2515
> Project: Apache Avro
>  Issue Type: Improvement
>  Components: java
>Affects Versions: 1.9.0
> Environment: N/A
>Reporter: Teodor Kostov
>Assignee: Teodor Kostov
>Priority: Minor
>  Labels: pull-request-available
> Fix For: 1.10.0, 1.9.1
>
>
> Enable direct access to utility methods in {{ProtobufData}} that deal with 
> the message description on a more granular level.
> {{ProtobufData}} presents some great functionality that allows dealing with 
> the Protobuf message descriptors. The intention of this PR is to provide 
> access to the methods that deal with these descriptors on a more granular 
> level. This would make it more flexible for the library clients to deal with 
> their Protobuf message.
> In terms of extending the functionality this would make it possible for 
> clients to adjust some specific moments of the conversion from Protobuf 
> descriptors to Avro schemas.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

Re: [VOTE] Release Apache Avro 1.9.1 RC1

2019-08-22 Thread Driesprong, Fokko

I'm fine with merging it into 1.9.1 since it is only the performance tests
and no core components of Avro. The impact will be minimal and I would like
to keep it Java11 compatible.

It would be nice to introduce some check to capture this in the future.

Cheers, Fokko

Op wo 21 aug. 2019 om 16:52 schreef Ismaël Mejía :

> Hello,
>
> I have a question, while checking some Java 11 improvements I
> discovered that the Perf module that we introduced in 1.9.0 has a
> split package issue that can cause conflicts with Java 11. I was
> wondering if we could fix this (the fix is just to isolate the module
> classes into its own package), however this is not backwards
> compatible with Avro 1.9.0. So should we get this into this release?
> or be strict and let it for 1.10.x?
>
> I just opened the PR for the solution for master (1.10.0) [1] but was
> thinking that it is probably worth to cherry pick it and include it in
> 1.9.1 even if breaking the previous version compatiblity, because it
> is quite new and I expect it has low impact.
>
> Any opinions?
>
> [1] https://github.com/apache/avro/pull/615
>
>
> https://issues.apache.org/jira/browse/AVRO-2517
>
> On Wed, Aug 21, 2019 at 11:22 AM Driesprong, Fokko 
> wrote:
> >
> > Thank you Brian,
> >
> > I'm canceling the RC1 and I'm doing some checks on the 1.9 branch. Seeing
> > some funky stuff with the RAT checker. I'll prepare for RC2 this evening.
> >
> > Cheers, Fokko
> >
> >
> >
> >
> > Op wo 21 aug. 2019 om 04:17 schreef Brian Lachniet  >:
> >
> > > It sounds like an rc2 is imminent, but I went ahead and ran some basic
> > > checks on the C# bindings. I didn't find anything unusual. I published
> > > Apache.Avro
> > > v1.9.1-rc1  and
> > > Apache.Avro.Tools
> > > v1.9.1-rc1 
> to
> > > nuget.org.
> > >
> > > On Tue, Aug 20, 2019 at 3:44 PM Driesprong, Fokko  >
> > > wrote:
> > >
> > > > Thanks, Dan for giving it a try. My mistake, this file comes from the
> > > > test-suite. I'll clean up the repo and publish a new package.
> > > >
> > > > Cheers, Fokko
> > > >
> > > > Op di 20 aug. 2019 om 21:41 schreef Daniel Kulp :
> > > >
> > > > > Java doesn’t build.There is an empty file:
> > > > >
> > > > >
> > > > >
> > > >
> > >
> lang/java/mapred/src/test/resources/org/apache/avro/mapreduce/mapreduce-test-input.avro/SUCCESS.crc
> > > > >
> > > > > In the tar that doesn’t seem to be in the git repo.  It’s causing a
> > > > > problem with RAT since there isn’t a license header or anything.
>  I’m
> > > > not
> > > > > sure where that file came from.
> > > > >
> > > > >
> > > > > Dan
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > > On Aug 20, 2019, at 2:04 PM, Driesprong, Fokko
>  > > >
> > > > > wrote:
> > > > > >
> > > > > > Hi everyone,
> > > > > >
> > > > > > I'm delighted to propose the following RC to be released as
> official
> > > > > Apache
> > > > > > Avro 1.9.1 release.
> > > > > >
> > > > > > The commit id is aad028bf84d43cc3481ac8b527f30debbdf213d2
> > > > > > * This corresponds to the tag: release-1.9.1-rc1
> > > > > > * https://github.com/apache/avro/releases/tag/release-1.9.1-rc1
> > > > > >
> > > > > > The release tarball, signature, and checksums are here:
> > > > > > * https://dist.apache.org/repos/dist/dev/avro/avro-1.9.1-rc1/
> > > > > >
> > > > > > You can find the KEYS file here:
> > > > > > * https://dist.apache.org/repos/dist/dev/avro/KEYS
> > > > > >
> > > > > > Binary artifacts for Java are staged in Nexus here:
> > > > > > *
> > > > > >
> > > > >
> > > >
> > >
> https://repository.apache.org/content/groups/staging/org/apache/avro/avro/1.9.1/
> > > > > >
> > > > > > This release includes 31 Jira issues:
> > > > > >
> > > > >
> > > >
> > >
> https://jira.apache.org/jira/issues/?jql=project%20%3D%20AVRO%20AND%20fixVersion%20%3D%201.9.1
> > > > > > * Most important, fix regression issues:
> > > > > >  * Java: decoding schema's:
> > > > > https://jira.apache.org/jira/browse/AVRO-2400
> > > > > >  * .Net: Performance issue:
> > > > > https://jira.apache.org/jira/browse/AVRO-2396
> > > > > > * Java: Make org.apache.avro.Schema serializable
> > > > > > * Java: Ability to add custom object to Velocity templating
> > > > > > * Improved interoperability testing
> > > > > > * Removed NPE's
> > > > > > * Upgrade dependencies to latest to the latest version
> > > > > > * And many more :-)
> > > > > >
> > > > > > Please download, verify, and test. This vote will remain open
> for at
> > > > > least
> > > > > > 72 hours. Given sufficient votes, I would like to close it on or
> > > about
> > > > > > midnight
> > > > > > on Saturday, 24 August 2019.
> > > > > >
> > > > > > [ ] +1 Release this as Apache Avro 1.9.1
> > > > > > [ ] +0
> > > > > > [ ] -1 Do not release this because...
> > > > > >
> > > > > > Consider this a +1 (binding) from my side:
> > > > > > * Compiled against Divolte collector and Iceberg
> > > > > >
> > > > > > Cheers, Fokko

[jira] [Resolved] (AVRO-2515) Enable direct access to utility methods in ProtobufData

2019-08-22 Thread Fokko Driesprong (Jira)



 [ 
https://issues.apache.org/jira/browse/AVRO-2515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Fokko Driesprong resolved AVRO-2515.

Resolution: Fixed

> Enable direct access to utility methods in ProtobufData
> ---
>
> Key: AVRO-2515
> URL: https://issues.apache.org/jira/browse/AVRO-2515
> Project: Apache Avro
>  Issue Type: Improvement
>  Components: java
>Affects Versions: 1.9.0
> Environment: N/A
>Reporter: Teodor Kostov
>Assignee: Teodor Kostov
>Priority: Minor
>  Labels: pull-request-available
> Fix For: 1.10.0, 1.9.1
>
>
> Enable direct access to utility methods in {{ProtobufData}} that deal with 
> the message description on a more granular level.
> {{ProtobufData}} presents some great functionality that allows dealing with 
> the Protobuf message descriptors. The intention of this PR is to provide 
> access to the methods that deal with these descriptors on a more granular 
> level. This would make it more flexible for the library clients to deal with 
> their Protobuf message.
> In terms of extending the functionality this would make it possible for 
> clients to adjust some specific moments of the conversion from Protobuf 
> descriptors to Avro schemas.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Assigned] (AVRO-2515) Enable direct access to utility methods in ProtobufData

2019-08-22 Thread Fokko Driesprong (Jira)



 [ 
https://issues.apache.org/jira/browse/AVRO-2515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Fokko Driesprong reassigned AVRO-2515:
--

Assignee: Teodor Kostov

> Enable direct access to utility methods in ProtobufData
> ---
>
> Key: AVRO-2515
> URL: https://issues.apache.org/jira/browse/AVRO-2515
> Project: Apache Avro
>  Issue Type: Improvement
>  Components: java
>Affects Versions: 1.9.0
> Environment: N/A
>Reporter: Teodor Kostov
>Assignee: Teodor Kostov
>Priority: Minor
>  Labels: pull-request-available
> Fix For: 1.10.0, 1.9.1
>
>
> Enable direct access to utility methods in {{ProtobufData}} that deal with 
> the message description on a more granular level.
> {{ProtobufData}} presents some great functionality that allows dealing with 
> the Protobuf message descriptors. The intention of this PR is to provide 
> access to the methods that deal with these descriptors on a more granular 
> level. This would make it more flexible for the library clients to deal with 
> their Protobuf message.
> In terms of extending the functionality this would make it possible for 
> clients to adjust some specific moments of the conversion from Protobuf 
> descriptors to Avro schemas.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Commented] (AVRO-2515) Enable direct access to utility methods in ProtobufData

2019-08-22 Thread ASF subversion and git services (Jira)



[ 
https://issues.apache.org/jira/browse/AVRO-2515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913237#comment-16913237
 ] 

ASF subversion and git services commented on AVRO-2515:
---

Commit 9f5209caeadd45d7a7787005ec1a106540f93c88 in avro's branch 
refs/heads/branch-1.9.1 from Teodor Kostov
[ https://gitbox.apache.org/repos/asf?p=avro.git;h=9f5209c ]

AVRO-2515: Expand access of ProtobufData methods (#612)

Chandged from private to public getSchema(FieldDescriptor) and
getNamespace(FileDescriptor, Descriptor).

The utility methods that deal with the conversion from Protobuf message
descriptor to an Avro schema are now exposed on a more granular level.
This will alow the library clients to use this functionality and not
reinvent it. This will also help with extending the base functionality.
For example, adding protobuf meta information to the schema as
properties.

(cherry picked from commit bce00d9c4a44f66c6906e66cde21938935c00fc5)


> Enable direct access to utility methods in ProtobufData
> ---
>
> Key: AVRO-2515
> URL: https://issues.apache.org/jira/browse/AVRO-2515
> Project: Apache Avro
>  Issue Type: Improvement
>  Components: java
>Affects Versions: 1.9.0
> Environment: N/A
>Reporter: Teodor Kostov
>Priority: Minor
>  Labels: pull-request-available
>
> Enable direct access to utility methods in {{ProtobufData}} that deal with 
> the message description on a more granular level.
> {{ProtobufData}} presents some great functionality that allows dealing with 
> the Protobuf message descriptors. The intention of this PR is to provide 
> access to the methods that deal with these descriptors on a more granular 
> level. This would make it more flexible for the library clients to deal with 
> their Protobuf message.
> In terms of extending the functionality this would make it possible for 
> clients to adjust some specific moments of the conversion from Protobuf 
> descriptors to Avro schemas.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Updated] (AVRO-2515) Enable direct access to utility methods in ProtobufData

2019-08-22 Thread Fokko Driesprong (Jira)



 [ 
https://issues.apache.org/jira/browse/AVRO-2515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Fokko Driesprong updated AVRO-2515:
---
Fix Version/s: 1.9.1
   1.10.0

> Enable direct access to utility methods in ProtobufData
> ---
>
> Key: AVRO-2515
> URL: https://issues.apache.org/jira/browse/AVRO-2515
> Project: Apache Avro
>  Issue Type: Improvement
>  Components: java
>Affects Versions: 1.9.0
> Environment: N/A
>Reporter: Teodor Kostov
>Priority: Minor
>  Labels: pull-request-available
> Fix For: 1.10.0, 1.9.1
>
>
> Enable direct access to utility methods in {{ProtobufData}} that deal with 
> the message description on a more granular level.
> {{ProtobufData}} presents some great functionality that allows dealing with 
> the Protobuf message descriptors. The intention of this PR is to provide 
> access to the methods that deal with these descriptors on a more granular 
> level. This would make it more flexible for the library clients to deal with 
> their Protobuf message.
> In terms of extending the functionality this would make it possible for 
> clients to adjust some specific moments of the conversion from Protobuf 
> descriptors to Avro schemas.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Commented] (AVRO-2515) Enable direct access to utility methods in ProtobufData

2019-08-22 Thread ASF subversion and git services (Jira)



[ 
https://issues.apache.org/jira/browse/AVRO-2515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913235#comment-16913235
 ] 

ASF subversion and git services commented on AVRO-2515:
---

Commit f8c22c8acccea720bb0baa0951a2586c29197461 in avro's branch 
refs/heads/branch-1.9 from Teodor Kostov
[ https://gitbox.apache.org/repos/asf?p=avro.git;h=f8c22c8 ]

AVRO-2515: Expand access of ProtobufData methods (#612)

Chandged from private to public getSchema(FieldDescriptor) and
getNamespace(FileDescriptor, Descriptor).

The utility methods that deal with the conversion from Protobuf message
descriptor to an Avro schema are now exposed on a more granular level.
This will alow the library clients to use this functionality and not
reinvent it. This will also help with extending the base functionality.
For example, adding protobuf meta information to the schema as
properties.

(cherry picked from commit bce00d9c4a44f66c6906e66cde21938935c00fc5)


> Enable direct access to utility methods in ProtobufData
> ---
>
> Key: AVRO-2515
> URL: https://issues.apache.org/jira/browse/AVRO-2515
> Project: Apache Avro
>  Issue Type: Improvement
>  Components: java
>Affects Versions: 1.9.0
> Environment: N/A
>Reporter: Teodor Kostov
>Priority: Minor
>  Labels: pull-request-available
>
> Enable direct access to utility methods in {{ProtobufData}} that deal with 
> the message description on a more granular level.
> {{ProtobufData}} presents some great functionality that allows dealing with 
> the Protobuf message descriptors. The intention of this PR is to provide 
> access to the methods that deal with these descriptors on a more granular 
> level. This would make it more flexible for the library clients to deal with 
> their Protobuf message.
> In terms of extending the functionality this would make it possible for 
> clients to adjust some specific moments of the conversion from Protobuf 
> descriptors to Avro schemas.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Commented] (AVRO-2515) Enable direct access to utility methods in ProtobufData

2019-08-22 Thread ASF subversion and git services (Jira)



[ 
https://issues.apache.org/jira/browse/AVRO-2515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913232#comment-16913232
 ] 

ASF subversion and git services commented on AVRO-2515:
---

Commit bce00d9c4a44f66c6906e66cde21938935c00fc5 in avro's branch 
refs/heads/master from Teodor Kostov
[ https://gitbox.apache.org/repos/asf?p=avro.git;h=bce00d9 ]

AVRO-2515: Expand access of ProtobufData methods (#612)

Chandged from private to public getSchema(FieldDescriptor) and
getNamespace(FileDescriptor, Descriptor).

The utility methods that deal with the conversion from Protobuf message
descriptor to an Avro schema are now exposed on a more granular level.
This will alow the library clients to use this functionality and not
reinvent it. This will also help with extending the base functionality.
For example, adding protobuf meta information to the schema as
properties.

> Enable direct access to utility methods in ProtobufData
> ---
>
> Key: AVRO-2515
> URL: https://issues.apache.org/jira/browse/AVRO-2515
> Project: Apache Avro
>  Issue Type: Improvement
>  Components: java
>Affects Versions: 1.9.0
> Environment: N/A
>Reporter: Teodor Kostov
>Priority: Minor
>  Labels: pull-request-available
>
> Enable direct access to utility methods in {{ProtobufData}} that deal with 
> the message description on a more granular level.
> {{ProtobufData}} presents some great functionality that allows dealing with 
> the Protobuf message descriptors. The intention of this PR is to provide 
> access to the methods that deal with these descriptors on a more granular 
> level. This would make it more flexible for the library clients to deal with 
> their Protobuf message.
> In terms of extending the functionality this would make it possible for 
> clients to adjust some specific moments of the conversion from Protobuf 
> descriptors to Avro schemas.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Created] (AVRO-2519) Use NettyTransceiver repeatedly connect to remote server which doesn't support current RPC or unavailable will cause " java.lang.OutOfMemoryError: Direct buffer memory"

[jira] [Updated] (AVRO-2518) Ruby Avro doesn't validate enum fields with default values

[jira] [Updated] (AVRO-2518) Ruby Avro doesn't validate enum fields with default values

[jira] [Updated] (AVRO-2518) Ruby Avro doesn't validate enum fields with default values

[jira] [Created] (AVRO-2518) Ruby Avro doesn't validate enum fields with default values

[jira] [Commented] (AVRO-2515) Enable direct access to utility methods in ProtobufData

Re: [VOTE] Release Apache Avro 1.9.1 RC1

[jira] [Resolved] (AVRO-2515) Enable direct access to utility methods in ProtobufData

[jira] [Assigned] (AVRO-2515) Enable direct access to utility methods in ProtobufData

[jira] [Commented] (AVRO-2515) Enable direct access to utility methods in ProtobufData

[jira] [Updated] (AVRO-2515) Enable direct access to utility methods in ProtobufData

[jira] [Commented] (AVRO-2515) Enable direct access to utility methods in ProtobufData

[jira] [Commented] (AVRO-2515) Enable direct access to utility methods in ProtobufData

13 matches

Site Navigation

Mail list logo

Footer information