[jira] [Resolved] (HADOOP-15389) Hadoop contains both Jackson 2.9.4 and 2.7.8 jars
[ https://issues.apache.org/jira/browse/HADOOP-15389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Chuyko resolved HADOOP-15389. Resolution: Invalid Sorry, cannot be reproduced anymore in clean workspace. > Hadoop contains both Jackson 2.9.4 and 2.7.8 jars > - > > Key: HADOOP-15389 > URL: https://issues.apache.org/jira/browse/HADOOP-15389 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 3.1.0, 3.2.0 >Reporter: Dmitry Chuyko >Priority: Blocker > > I build hadoop-3.2.0-SNAPSHOT distribution from scratch. Resulting package > has following in hadoop-3.2.0-SNAPSHOT/share/hadoop/hdfs/lib/: > jackson-annotations-2.7.8.jar > jackson-annotations-2.9.4.jar > jackson-core-2.7.8.jar > jackson-core-2.9.4.jar > jackson-databind-2.7.8.jar > jackson-databind-2.9.4.jar > As a result DataNode does not start with following error: > java.lang.NoSuchFieldError: ACCEPT_CASE_INSENSITIVE_PROPERTIES > at > com.fasterxml.jackson.databind.deser.BeanDeserializerBase.createContextual(BeanDeserializerBase.java:747) > at > com.fasterxml.jackson.databind.DeserializationContext.handleSecondaryContextualization(DeserializationContext.java:682) > at > com.fasterxml.jackson.databind.DeserializationContext.findRootValueDeserializer(DeserializationContext.java:482) > at > com.fasterxml.jackson.databind.ObjectReader._prefetchRootDeserializer(ObjectReader.java:1938) > at com.fasterxml.jackson.databind.ObjectReader.(ObjectReader.java:189) > at > com.fasterxml.jackson.databind.ObjectMapper._newReader(ObjectMapper.java:658) > at > com.fasterxml.jackson.databind.ObjectMapper.readerFor(ObjectMapper.java:3517) > at > org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsVolumeImpl.(FsVolumeImpl.java:109) > at > org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsVolumeImplBuilder.build(FsVolumeImplBuilder.java:76) > at > org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.addVolume(FsDatasetImpl.java:426) > at > org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.(FsDatasetImpl.java:316) > at > org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetFactory.newInstance(FsDatasetFactory.java:34) > at > org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetFactory.newInstance(FsDatasetFactory.java:30) > at > org.apache.hadoop.hdfs.server.datanode.DataNode.initStorage(DataNode.java:1719) > at > org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:1665) > at > org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:390) > at > org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:280) > at > org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:816) > at java.lang.Thread.run(Thread.java:748) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-15389) Hadoop contains both Jackson 2.9.4 and 2.7.8 jars
Dmitry Chuyko created HADOOP-15389: -- Summary: Hadoop contains both Jackson 2.9.4 and 2.7.8 jars Key: HADOOP-15389 URL: https://issues.apache.org/jira/browse/HADOOP-15389 Project: Hadoop Common Issue Type: Bug Affects Versions: 3.1.0, 3.2.0 Reporter: Dmitry Chuyko I build hadoop-3.2.0-SNAPSHOT distribution from scratch. Resulting package has following in hadoop-3.2.0-SNAPSHOT/share/hadoop/hdfs/lib/: jackson-annotations-2.7.8.jar jackson-annotations-2.9.4.jar jackson-core-2.7.8.jar jackson-core-2.9.4.jar jackson-databind-2.7.8.jar jackson-databind-2.9.4.jar As a result DataNode does not start with following error: java.lang.NoSuchFieldError: ACCEPT_CASE_INSENSITIVE_PROPERTIES at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.createContextual(BeanDeserializerBase.java:747) at com.fasterxml.jackson.databind.DeserializationContext.handleSecondaryContextualization(DeserializationContext.java:682) at com.fasterxml.jackson.databind.DeserializationContext.findRootValueDeserializer(DeserializationContext.java:482) at com.fasterxml.jackson.databind.ObjectReader._prefetchRootDeserializer(ObjectReader.java:1938) at com.fasterxml.jackson.databind.ObjectReader.(ObjectReader.java:189) at com.fasterxml.jackson.databind.ObjectMapper._newReader(ObjectMapper.java:658) at com.fasterxml.jackson.databind.ObjectMapper.readerFor(ObjectMapper.java:3517) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsVolumeImpl.(FsVolumeImpl.java:109) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsVolumeImplBuilder.build(FsVolumeImplBuilder.java:76) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.addVolume(FsDatasetImpl.java:426) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.(FsDatasetImpl.java:316) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetFactory.newInstance(FsDatasetFactory.java:34) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetFactory.newInstance(FsDatasetFactory.java:30) at org.apache.hadoop.hdfs.server.datanode.DataNode.initStorage(DataNode.java:1719) at org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:1665) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:390) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:280) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:816) at java.lang.Thread.run(Thread.java:748) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15033) Use java.util.zip.CRC32C for Java 9 and above
[ https://issues.apache.org/jira/browse/HADOOP-15033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Chuyko updated HADOOP-15033: --- Attachment: HADOOP-15033.010.patch > Use java.util.zip.CRC32C for Java 9 and above > - > > Key: HADOOP-15033 > URL: https://issues.apache.org/jira/browse/HADOOP-15033 > Project: Hadoop Common > Issue Type: Improvement > Components: performance, util >Affects Versions: 3.0.0 >Reporter: Dmitry Chuyko > Labels: Java9, common, jdk9 > Attachments: HADOOP-15033.001.patch, HADOOP-15033.001.patch, > HADOOP-15033.002.patch, HADOOP-15033.003.patch, HADOOP-15033.003.patch, > HADOOP-15033.004.patch, HADOOP-15033.005.diff, HADOOP-15033.005.diff, > HADOOP-15033.005.diff, HADOOP-15033.005.patch, HADOOP-15033.006.patch, > HADOOP-15033.007.patch, HADOOP-15033.007.patch, HADOOP-15033.008.patch, > HADOOP-15033.009.patch, HADOOP-15033.009.patch, HADOOP-15033.009.patch, > HADOOP-15033.009.patch, HADOOP-15033.009.patch, HADOOP-15033.009.patch, > HADOOP-15033.009.patch, HADOOP-15033.010.patch > > > java.util.zip.CRC32C implementation is available since Java 9. > https://docs.oracle.com/javase/9/docs/api/java/util/zip/CRC32C.html > Platform specific assembler intrinsics make it more effective than any pure > Java implementation. > Hadoop is compiled against Java 8 but class constructor may be accessible > with method handle on 9 to instances implementing Checksum in runtime. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15033) Use java.util.zip.CRC32C for Java 9 and above
[ https://issues.apache.org/jira/browse/HADOOP-15033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16314787#comment-16314787 ] Dmitry Chuyko commented on HADOOP-15033: Attached HADOOP-15033.009.patch is a copy of https://patch-diff.githubusercontent.com/raw/apache/hadoop/pull/291.patch after changes from fist review round and after rebase to latest trunk. Testing was blocked for a long time by HADOOP-15128. Now the patch passed genericqa checks and shows same performance improvement a before. Please review this latest version of pull request. > Use java.util.zip.CRC32C for Java 9 and above > - > > Key: HADOOP-15033 > URL: https://issues.apache.org/jira/browse/HADOOP-15033 > Project: Hadoop Common > Issue Type: Improvement > Components: performance, util >Affects Versions: 3.0.0 >Reporter: Dmitry Chuyko > Labels: Java9, common, jdk9 > Attachments: HADOOP-15033.001.patch, HADOOP-15033.001.patch, > HADOOP-15033.002.patch, HADOOP-15033.003.patch, HADOOP-15033.003.patch, > HADOOP-15033.004.patch, HADOOP-15033.005.diff, HADOOP-15033.005.diff, > HADOOP-15033.005.diff, HADOOP-15033.005.patch, HADOOP-15033.006.patch, > HADOOP-15033.007.patch, HADOOP-15033.007.patch, HADOOP-15033.008.patch, > HADOOP-15033.009.patch, HADOOP-15033.009.patch, HADOOP-15033.009.patch, > HADOOP-15033.009.patch, HADOOP-15033.009.patch, HADOOP-15033.009.patch, > HADOOP-15033.009.patch > > > java.util.zip.CRC32C implementation is available since Java 9. > https://docs.oracle.com/javase/9/docs/api/java/util/zip/CRC32C.html > Platform specific assembler intrinsics make it more effective than any pure > Java implementation. > Hadoop is compiled against Java 8 but class constructor may be accessible > with method handle on 9 to instances implementing Checksum in runtime. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15033) Use java.util.zip.CRC32C for Java 9 and above
[ https://issues.apache.org/jira/browse/HADOOP-15033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Chuyko updated HADOOP-15033: --- Attachment: HADOOP-15033.009.patch > Use java.util.zip.CRC32C for Java 9 and above > - > > Key: HADOOP-15033 > URL: https://issues.apache.org/jira/browse/HADOOP-15033 > Project: Hadoop Common > Issue Type: Improvement > Components: performance, util >Affects Versions: 3.0.0 >Reporter: Dmitry Chuyko > Labels: Java9, common, jdk9 > Attachments: HADOOP-15033.001.patch, HADOOP-15033.001.patch, > HADOOP-15033.002.patch, HADOOP-15033.003.patch, HADOOP-15033.003.patch, > HADOOP-15033.004.patch, HADOOP-15033.005.diff, HADOOP-15033.005.diff, > HADOOP-15033.005.diff, HADOOP-15033.005.patch, HADOOP-15033.006.patch, > HADOOP-15033.007.patch, HADOOP-15033.007.patch, HADOOP-15033.008.patch, > HADOOP-15033.009.patch, HADOOP-15033.009.patch, HADOOP-15033.009.patch, > HADOOP-15033.009.patch, HADOOP-15033.009.patch, HADOOP-15033.009.patch, > HADOOP-15033.009.patch > > > java.util.zip.CRC32C implementation is available since Java 9. > https://docs.oracle.com/javase/9/docs/api/java/util/zip/CRC32C.html > Platform specific assembler intrinsics make it more effective than any pure > Java implementation. > Hadoop is compiled against Java 8 but class constructor may be accessible > with method handle on 9 to instances implementing Checksum in runtime. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15033) Use java.util.zip.CRC32C for Java 9 and above
[ https://issues.apache.org/jira/browse/HADOOP-15033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Chuyko updated HADOOP-15033: --- Attachment: HADOOP-15033.009.patch > Use java.util.zip.CRC32C for Java 9 and above > - > > Key: HADOOP-15033 > URL: https://issues.apache.org/jira/browse/HADOOP-15033 > Project: Hadoop Common > Issue Type: Improvement > Components: performance, util >Affects Versions: 3.0.0 >Reporter: Dmitry Chuyko > Labels: Java9, common, jdk9 > Attachments: HADOOP-15033.001.patch, HADOOP-15033.001.patch, > HADOOP-15033.002.patch, HADOOP-15033.003.patch, HADOOP-15033.003.patch, > HADOOP-15033.004.patch, HADOOP-15033.005.diff, HADOOP-15033.005.diff, > HADOOP-15033.005.diff, HADOOP-15033.005.patch, HADOOP-15033.006.patch, > HADOOP-15033.007.patch, HADOOP-15033.007.patch, HADOOP-15033.008.patch, > HADOOP-15033.009.patch, HADOOP-15033.009.patch, HADOOP-15033.009.patch, > HADOOP-15033.009.patch, HADOOP-15033.009.patch, HADOOP-15033.009.patch > > > java.util.zip.CRC32C implementation is available since Java 9. > https://docs.oracle.com/javase/9/docs/api/java/util/zip/CRC32C.html > Platform specific assembler intrinsics make it more effective than any pure > Java implementation. > Hadoop is compiled against Java 8 but class constructor may be accessible > with method handle on 9 to instances implementing Checksum in runtime. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15033) Use java.util.zip.CRC32C for Java 9 and above
[ https://issues.apache.org/jira/browse/HADOOP-15033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Chuyko updated HADOOP-15033: --- Attachment: HADOOP-15033.009.patch > Use java.util.zip.CRC32C for Java 9 and above > - > > Key: HADOOP-15033 > URL: https://issues.apache.org/jira/browse/HADOOP-15033 > Project: Hadoop Common > Issue Type: Improvement > Components: performance, util >Affects Versions: 3.0.0 >Reporter: Dmitry Chuyko > Labels: Java9, common, jdk9 > Attachments: HADOOP-15033.001.patch, HADOOP-15033.001.patch, > HADOOP-15033.002.patch, HADOOP-15033.003.patch, HADOOP-15033.003.patch, > HADOOP-15033.004.patch, HADOOP-15033.005.diff, HADOOP-15033.005.diff, > HADOOP-15033.005.diff, HADOOP-15033.005.patch, HADOOP-15033.006.patch, > HADOOP-15033.007.patch, HADOOP-15033.007.patch, HADOOP-15033.008.patch, > HADOOP-15033.009.patch, HADOOP-15033.009.patch, HADOOP-15033.009.patch, > HADOOP-15033.009.patch, HADOOP-15033.009.patch > > > java.util.zip.CRC32C implementation is available since Java 9. > https://docs.oracle.com/javase/9/docs/api/java/util/zip/CRC32C.html > Platform specific assembler intrinsics make it more effective than any pure > Java implementation. > Hadoop is compiled against Java 8 but class constructor may be accessible > with method handle on 9 to instances implementing Checksum in runtime. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15033) Use java.util.zip.CRC32C for Java 9 and above
[ https://issues.apache.org/jira/browse/HADOOP-15033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Chuyko updated HADOOP-15033: --- Attachment: HADOOP-15033.009.patch > Use java.util.zip.CRC32C for Java 9 and above > - > > Key: HADOOP-15033 > URL: https://issues.apache.org/jira/browse/HADOOP-15033 > Project: Hadoop Common > Issue Type: Improvement > Components: performance, util >Affects Versions: 3.0.0 >Reporter: Dmitry Chuyko > Labels: Java9, common, jdk9 > Attachments: HADOOP-15033.001.patch, HADOOP-15033.001.patch, > HADOOP-15033.002.patch, HADOOP-15033.003.patch, HADOOP-15033.003.patch, > HADOOP-15033.004.patch, HADOOP-15033.005.diff, HADOOP-15033.005.diff, > HADOOP-15033.005.diff, HADOOP-15033.005.patch, HADOOP-15033.006.patch, > HADOOP-15033.007.patch, HADOOP-15033.007.patch, HADOOP-15033.008.patch, > HADOOP-15033.009.patch, HADOOP-15033.009.patch, HADOOP-15033.009.patch, > HADOOP-15033.009.patch > > > java.util.zip.CRC32C implementation is available since Java 9. > https://docs.oracle.com/javase/9/docs/api/java/util/zip/CRC32C.html > Platform specific assembler intrinsics make it more effective than any pure > Java implementation. > Hadoop is compiled against Java 8 but class constructor may be accessible > with method handle on 9 to instances implementing Checksum in runtime. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15033) Use java.util.zip.CRC32C for Java 9 and above
[ https://issues.apache.org/jira/browse/HADOOP-15033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Chuyko updated HADOOP-15033: --- Attachment: HADOOP-15033.009.patch > Use java.util.zip.CRC32C for Java 9 and above > - > > Key: HADOOP-15033 > URL: https://issues.apache.org/jira/browse/HADOOP-15033 > Project: Hadoop Common > Issue Type: Improvement > Components: performance, util >Affects Versions: 3.0.0 >Reporter: Dmitry Chuyko > Labels: Java9, common, jdk9 > Attachments: HADOOP-15033.001.patch, HADOOP-15033.001.patch, > HADOOP-15033.002.patch, HADOOP-15033.003.patch, HADOOP-15033.003.patch, > HADOOP-15033.004.patch, HADOOP-15033.005.diff, HADOOP-15033.005.diff, > HADOOP-15033.005.diff, HADOOP-15033.005.patch, HADOOP-15033.006.patch, > HADOOP-15033.007.patch, HADOOP-15033.007.patch, HADOOP-15033.008.patch, > HADOOP-15033.009.patch, HADOOP-15033.009.patch, HADOOP-15033.009.patch > > > java.util.zip.CRC32C implementation is available since Java 9. > https://docs.oracle.com/javase/9/docs/api/java/util/zip/CRC32C.html > Platform specific assembler intrinsics make it more effective than any pure > Java implementation. > Hadoop is compiled against Java 8 but class constructor may be accessible > with method handle on 9 to instances implementing Checksum in runtime. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15033) Use java.util.zip.CRC32C for Java 9 and above
[ https://issues.apache.org/jira/browse/HADOOP-15033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Chuyko updated HADOOP-15033: --- Attachment: HADOOP-15033.009.patch > Use java.util.zip.CRC32C for Java 9 and above > - > > Key: HADOOP-15033 > URL: https://issues.apache.org/jira/browse/HADOOP-15033 > Project: Hadoop Common > Issue Type: Improvement > Components: performance, util >Affects Versions: 3.0.0 >Reporter: Dmitry Chuyko > Labels: Java9, common, jdk9 > Attachments: HADOOP-15033.001.patch, HADOOP-15033.001.patch, > HADOOP-15033.002.patch, HADOOP-15033.003.patch, HADOOP-15033.003.patch, > HADOOP-15033.004.patch, HADOOP-15033.005.diff, HADOOP-15033.005.diff, > HADOOP-15033.005.diff, HADOOP-15033.005.patch, HADOOP-15033.006.patch, > HADOOP-15033.007.patch, HADOOP-15033.007.patch, HADOOP-15033.008.patch, > HADOOP-15033.009.patch, HADOOP-15033.009.patch > > > java.util.zip.CRC32C implementation is available since Java 9. > https://docs.oracle.com/javase/9/docs/api/java/util/zip/CRC32C.html > Platform specific assembler intrinsics make it more effective than any pure > Java implementation. > Hadoop is compiled against Java 8 but class constructor may be accessible > with method handle on 9 to instances implementing Checksum in runtime. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15033) Use java.util.zip.CRC32C for Java 9 and above
[ https://issues.apache.org/jira/browse/HADOOP-15033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Chuyko updated HADOOP-15033: --- Attachment: HADOOP-15033.009.patch > Use java.util.zip.CRC32C for Java 9 and above > - > > Key: HADOOP-15033 > URL: https://issues.apache.org/jira/browse/HADOOP-15033 > Project: Hadoop Common > Issue Type: Improvement > Components: performance, util >Affects Versions: 3.0.0 >Reporter: Dmitry Chuyko > Labels: Java9, common, jdk9 > Attachments: HADOOP-15033.001.patch, HADOOP-15033.001.patch, > HADOOP-15033.002.patch, HADOOP-15033.003.patch, HADOOP-15033.003.patch, > HADOOP-15033.004.patch, HADOOP-15033.005.diff, HADOOP-15033.005.diff, > HADOOP-15033.005.diff, HADOOP-15033.005.patch, HADOOP-15033.006.patch, > HADOOP-15033.007.patch, HADOOP-15033.007.patch, HADOOP-15033.008.patch, > HADOOP-15033.009.patch > > > java.util.zip.CRC32C implementation is available since Java 9. > https://docs.oracle.com/javase/9/docs/api/java/util/zip/CRC32C.html > Platform specific assembler intrinsics make it more effective than any pure > Java implementation. > Hadoop is compiled against Java 8 but class constructor may be accessible > with method handle on 9 to instances implementing Checksum in runtime. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15033) Use java.util.zip.CRC32C for Java 9 and above
[ https://issues.apache.org/jira/browse/HADOOP-15033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Chuyko updated HADOOP-15033: --- Attachment: HADOOP-15033.008.patch > Use java.util.zip.CRC32C for Java 9 and above > - > > Key: HADOOP-15033 > URL: https://issues.apache.org/jira/browse/HADOOP-15033 > Project: Hadoop Common > Issue Type: Improvement > Components: performance, util >Affects Versions: 3.0.0 >Reporter: Dmitry Chuyko > Labels: Java9, common, jdk9 > Attachments: HADOOP-15033.001.patch, HADOOP-15033.001.patch, > HADOOP-15033.002.patch, HADOOP-15033.003.patch, HADOOP-15033.003.patch, > HADOOP-15033.004.patch, HADOOP-15033.005.diff, HADOOP-15033.005.diff, > HADOOP-15033.005.diff, HADOOP-15033.005.patch, HADOOP-15033.006.patch, > HADOOP-15033.007.patch, HADOOP-15033.007.patch, HADOOP-15033.008.patch > > > java.util.zip.CRC32C implementation is available since Java 9. > https://docs.oracle.com/javase/9/docs/api/java/util/zip/CRC32C.html > Platform specific assembler intrinsics make it more effective than any pure > Java implementation. > Hadoop is compiled against Java 8 but class constructor may be accessible > with method handle on 9 to instances implementing Checksum in runtime. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15033) Use java.util.zip.CRC32C for Java 9 and above
[ https://issues.apache.org/jira/browse/HADOOP-15033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Chuyko updated HADOOP-15033: --- Attachment: HADOOP-15033.007.patch > Use java.util.zip.CRC32C for Java 9 and above > - > > Key: HADOOP-15033 > URL: https://issues.apache.org/jira/browse/HADOOP-15033 > Project: Hadoop Common > Issue Type: Improvement > Components: performance, util >Affects Versions: 3.0.0 >Reporter: Dmitry Chuyko > Labels: Java9, common, jdk9 > Attachments: HADOOP-15033.001.patch, HADOOP-15033.001.patch, > HADOOP-15033.002.patch, HADOOP-15033.003.patch, HADOOP-15033.003.patch, > HADOOP-15033.004.patch, HADOOP-15033.005.diff, HADOOP-15033.005.diff, > HADOOP-15033.005.diff, HADOOP-15033.005.patch, HADOOP-15033.006.patch, > HADOOP-15033.007.patch, HADOOP-15033.007.patch > > > java.util.zip.CRC32C implementation is available since Java 9. > https://docs.oracle.com/javase/9/docs/api/java/util/zip/CRC32C.html > Platform specific assembler intrinsics make it more effective than any pure > Java implementation. > Hadoop is compiled against Java 8 but class constructor may be accessible > with method handle on 9 to instances implementing Checksum in runtime. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15033) Use java.util.zip.CRC32C for Java 9 and above
[ https://issues.apache.org/jira/browse/HADOOP-15033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Chuyko updated HADOOP-15033: --- Attachment: HADOOP-15033.007.patch > Use java.util.zip.CRC32C for Java 9 and above > - > > Key: HADOOP-15033 > URL: https://issues.apache.org/jira/browse/HADOOP-15033 > Project: Hadoop Common > Issue Type: Improvement > Components: performance, util >Affects Versions: 3.0.0 >Reporter: Dmitry Chuyko > Labels: Java9, common, jdk9 > Attachments: HADOOP-15033.001.patch, HADOOP-15033.001.patch, > HADOOP-15033.002.patch, HADOOP-15033.003.patch, HADOOP-15033.003.patch, > HADOOP-15033.004.patch, HADOOP-15033.005.diff, HADOOP-15033.005.diff, > HADOOP-15033.005.diff, HADOOP-15033.005.patch, HADOOP-15033.006.patch, > HADOOP-15033.007.patch > > > java.util.zip.CRC32C implementation is available since Java 9. > https://docs.oracle.com/javase/9/docs/api/java/util/zip/CRC32C.html > Platform specific assembler intrinsics make it more effective than any pure > Java implementation. > Hadoop is compiled against Java 8 but class constructor may be accessible > with method handle on 9 to instances implementing Checksum in runtime. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15033) Use java.util.zip.CRC32C for Java 9 and above
[ https://issues.apache.org/jira/browse/HADOOP-15033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Chuyko updated HADOOP-15033: --- Attachment: HADOOP-15033.006.patch > Use java.util.zip.CRC32C for Java 9 and above > - > > Key: HADOOP-15033 > URL: https://issues.apache.org/jira/browse/HADOOP-15033 > Project: Hadoop Common > Issue Type: Improvement > Components: performance, util >Affects Versions: 3.0.0 >Reporter: Dmitry Chuyko > Labels: Java9, common, jdk9 > Attachments: HADOOP-15033.001.patch, HADOOP-15033.001.patch, > HADOOP-15033.002.patch, HADOOP-15033.003.patch, HADOOP-15033.003.patch, > HADOOP-15033.004.patch, HADOOP-15033.005.diff, HADOOP-15033.005.diff, > HADOOP-15033.005.diff, HADOOP-15033.005.patch, HADOOP-15033.006.patch > > > java.util.zip.CRC32C implementation is available since Java 9. > https://docs.oracle.com/javase/9/docs/api/java/util/zip/CRC32C.html > Platform specific assembler intrinsics make it more effective than any pure > Java implementation. > Hadoop is compiled against Java 8 but class constructor may be accessible > with method handle on 9 to instances implementing Checksum in runtime. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15033) Use java.util.zip.CRC32C for Java 9 and above
[ https://issues.apache.org/jira/browse/HADOOP-15033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Chuyko updated HADOOP-15033: --- Attachment: HADOOP-15033.005.diff > Use java.util.zip.CRC32C for Java 9 and above > - > > Key: HADOOP-15033 > URL: https://issues.apache.org/jira/browse/HADOOP-15033 > Project: Hadoop Common > Issue Type: Improvement > Components: performance, util >Affects Versions: 3.0.0 >Reporter: Dmitry Chuyko > Labels: Java9, common, jdk9 > Attachments: HADOOP-15033.001.patch, HADOOP-15033.001.patch, > HADOOP-15033.002.patch, HADOOP-15033.003.patch, HADOOP-15033.003.patch, > HADOOP-15033.004.patch, HADOOP-15033.005.diff, HADOOP-15033.005.diff, > HADOOP-15033.005.diff, HADOOP-15033.005.patch > > > java.util.zip.CRC32C implementation is available since Java 9. > https://docs.oracle.com/javase/9/docs/api/java/util/zip/CRC32C.html > Platform specific assembler intrinsics make it more effective than any pure > Java implementation. > Hadoop is compiled against Java 8 but class constructor may be accessible > with method handle on 9 to instances implementing Checksum in runtime. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15033) Use java.util.zip.CRC32C for Java 9 and above
[ https://issues.apache.org/jira/browse/HADOOP-15033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Chuyko updated HADOOP-15033: --- Attachment: HADOOP-15033.005.diff > Use java.util.zip.CRC32C for Java 9 and above > - > > Key: HADOOP-15033 > URL: https://issues.apache.org/jira/browse/HADOOP-15033 > Project: Hadoop Common > Issue Type: Improvement > Components: performance, util >Affects Versions: 3.0.0 >Reporter: Dmitry Chuyko > Labels: Java9, common, jdk9 > Attachments: HADOOP-15033.001.patch, HADOOP-15033.001.patch, > HADOOP-15033.002.patch, HADOOP-15033.003.patch, HADOOP-15033.003.patch, > HADOOP-15033.004.patch, HADOOP-15033.005.diff, HADOOP-15033.005.diff, > HADOOP-15033.005.patch > > > java.util.zip.CRC32C implementation is available since Java 9. > https://docs.oracle.com/javase/9/docs/api/java/util/zip/CRC32C.html > Platform specific assembler intrinsics make it more effective than any pure > Java implementation. > Hadoop is compiled against Java 8 but class constructor may be accessible > with method handle on 9 to instances implementing Checksum in runtime. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15033) Use java.util.zip.CRC32C for Java 9 and above
[ https://issues.apache.org/jira/browse/HADOOP-15033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Chuyko updated HADOOP-15033: --- Attachment: HADOOP-15033.005.diff > Use java.util.zip.CRC32C for Java 9 and above > - > > Key: HADOOP-15033 > URL: https://issues.apache.org/jira/browse/HADOOP-15033 > Project: Hadoop Common > Issue Type: Improvement > Components: performance, util >Affects Versions: 3.0.0 >Reporter: Dmitry Chuyko > Labels: Java9, common, jdk9 > Attachments: HADOOP-15033.001.patch, HADOOP-15033.001.patch, > HADOOP-15033.002.patch, HADOOP-15033.003.patch, HADOOP-15033.003.patch, > HADOOP-15033.004.patch, HADOOP-15033.005.diff, HADOOP-15033.005.patch > > > java.util.zip.CRC32C implementation is available since Java 9. > https://docs.oracle.com/javase/9/docs/api/java/util/zip/CRC32C.html > Platform specific assembler intrinsics make it more effective than any pure > Java implementation. > Hadoop is compiled against Java 8 but class constructor may be accessible > with method handle on 9 to instances implementing Checksum in runtime. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15033) Use java.util.zip.CRC32C for Java 9 and above
[ https://issues.apache.org/jira/browse/HADOOP-15033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Chuyko updated HADOOP-15033: --- Attachment: HADOOP-15033.005.patch Attached HADOOP-15033.005.patch > Use java.util.zip.CRC32C for Java 9 and above > - > > Key: HADOOP-15033 > URL: https://issues.apache.org/jira/browse/HADOOP-15033 > Project: Hadoop Common > Issue Type: Improvement > Components: performance, util >Affects Versions: 3.0.0 >Reporter: Dmitry Chuyko > Labels: Java9, common, jdk9 > Attachments: HADOOP-15033.001.patch, HADOOP-15033.001.patch, > HADOOP-15033.002.patch, HADOOP-15033.003.patch, HADOOP-15033.003.patch, > HADOOP-15033.004.patch, HADOOP-15033.005.patch > > > java.util.zip.CRC32C implementation is available since Java 9. > https://docs.oracle.com/javase/9/docs/api/java/util/zip/CRC32C.html > Platform specific assembler intrinsics make it more effective than any pure > Java implementation. > Hadoop is compiled against Java 8 but class constructor may be accessible > with method handle on 9 to instances implementing Checksum in runtime. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15033) Use java.util.zip.CRC32C for Java 9 and above
[ https://issues.apache.org/jira/browse/HADOOP-15033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16262782#comment-16262782 ] Dmitry Chuyko commented on HADOOP-15033: So attached HADOOP-15033.004.patch which is a copy of https://patch-diff.githubusercontent.com/raw/apache/hadoop/pull/291.patch passes pre-commit QA checks and shows ~4x improvement in benchmarks. Could someone please review it? > Use java.util.zip.CRC32C for Java 9 and above > - > > Key: HADOOP-15033 > URL: https://issues.apache.org/jira/browse/HADOOP-15033 > Project: Hadoop Common > Issue Type: Improvement > Components: performance, util >Affects Versions: 3.0.0 >Reporter: Dmitry Chuyko > Attachments: HADOOP-15033.001.patch, HADOOP-15033.001.patch, > HADOOP-15033.002.patch, HADOOP-15033.003.patch, HADOOP-15033.003.patch, > HADOOP-15033.004.patch > > > java.util.zip.CRC32C implementation is available since Java 9. > https://docs.oracle.com/javase/9/docs/api/java/util/zip/CRC32C.html > Platform specific assembler intrinsics make it more effective than any pure > Java implementation. > Hadoop is compiled against Java 8 but class constructor may be accessible > with method handle on 9 to instances implementing Checksum in runtime. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15033) Use java.util.zip.CRC32C for Java 9 and above
[ https://issues.apache.org/jira/browse/HADOOP-15033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Chuyko updated HADOOP-15033: --- Attachment: HADOOP-15033.004.patch > Use java.util.zip.CRC32C for Java 9 and above > - > > Key: HADOOP-15033 > URL: https://issues.apache.org/jira/browse/HADOOP-15033 > Project: Hadoop Common > Issue Type: Improvement > Components: performance, util >Affects Versions: 3.0.0 >Reporter: Dmitry Chuyko > Attachments: HADOOP-15033.001.patch, HADOOP-15033.001.patch, > HADOOP-15033.002.patch, HADOOP-15033.003.patch, HADOOP-15033.003.patch, > HADOOP-15033.004.patch > > > java.util.zip.CRC32C implementation is available since Java 9. > https://docs.oracle.com/javase/9/docs/api/java/util/zip/CRC32C.html > Platform specific assembler intrinsics make it more effective than any pure > Java implementation. > Hadoop is compiled against Java 8 but class constructor may be accessible > with method handle on 9 to instances implementing Checksum in runtime. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15033) Use java.util.zip.CRC32C for Java 9 and above
[ https://issues.apache.org/jira/browse/HADOOP-15033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Chuyko updated HADOOP-15033: --- Attachment: HADOOP-15033.003.patch > Use java.util.zip.CRC32C for Java 9 and above > - > > Key: HADOOP-15033 > URL: https://issues.apache.org/jira/browse/HADOOP-15033 > Project: Hadoop Common > Issue Type: Improvement > Components: performance, util >Affects Versions: 3.0.0 >Reporter: Dmitry Chuyko > Attachments: HADOOP-15033.001.patch, HADOOP-15033.001.patch, > HADOOP-15033.002.patch, HADOOP-15033.003.patch, HADOOP-15033.003.patch > > > java.util.zip.CRC32C implementation is available since Java 9. > https://docs.oracle.com/javase/9/docs/api/java/util/zip/CRC32C.html > Platform specific assembler intrinsics make it more effective than any pure > Java implementation. > Hadoop is compiled against Java 8 but class constructor may be accessible > with method handle on 9 to instances implementing Checksum in runtime. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15033) Use java.util.zip.CRC32C for Java 9 and above
[ https://issues.apache.org/jira/browse/HADOOP-15033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Chuyko updated HADOOP-15033: --- Attachment: HADOOP-15033.003.patch > Use java.util.zip.CRC32C for Java 9 and above > - > > Key: HADOOP-15033 > URL: https://issues.apache.org/jira/browse/HADOOP-15033 > Project: Hadoop Common > Issue Type: Improvement > Components: performance, util >Affects Versions: 3.0.0 >Reporter: Dmitry Chuyko > Attachments: HADOOP-15033.001.patch, HADOOP-15033.001.patch, > HADOOP-15033.002.patch, HADOOP-15033.003.patch > > > java.util.zip.CRC32C implementation is available since Java 9. > https://docs.oracle.com/javase/9/docs/api/java/util/zip/CRC32C.html > Platform specific assembler intrinsics make it more effective than any pure > Java implementation. > Hadoop is compiled against Java 8 but class constructor may be accessible > with method handle on 9 to instances implementing Checksum in runtime. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15033) Use java.util.zip.CRC32C for Java 9 and above
[ https://issues.apache.org/jira/browse/HADOOP-15033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Chuyko updated HADOOP-15033: --- Attachment: HADOOP-15033.002.patch > Use java.util.zip.CRC32C for Java 9 and above > - > > Key: HADOOP-15033 > URL: https://issues.apache.org/jira/browse/HADOOP-15033 > Project: Hadoop Common > Issue Type: Improvement > Components: performance, util >Affects Versions: 3.0.0 >Reporter: Dmitry Chuyko > Attachments: HADOOP-15033.001.patch, HADOOP-15033.001.patch, > HADOOP-15033.002.patch > > > java.util.zip.CRC32C implementation is available since Java 9. > https://docs.oracle.com/javase/9/docs/api/java/util/zip/CRC32C.html > Platform specific assembler intrinsics make it more effective than any pure > Java implementation. > Hadoop is compiled against Java 8 but class constructor may be accessible > with method handle on 9 to instances implementing Checksum in runtime. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15033) Use java.util.zip.CRC32C for Java 9 and above
[ https://issues.apache.org/jira/browse/HADOOP-15033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Chuyko updated HADOOP-15033: --- Attachment: HADOOP-15033.001.patch > Use java.util.zip.CRC32C for Java 9 and above > - > > Key: HADOOP-15033 > URL: https://issues.apache.org/jira/browse/HADOOP-15033 > Project: Hadoop Common > Issue Type: Improvement > Components: performance, util >Affects Versions: 3.0.0 >Reporter: Dmitry Chuyko > Attachments: HADOOP-15033.001.patch, HADOOP-15033.001.patch > > > java.util.zip.CRC32C implementation is available since Java 9. > https://docs.oracle.com/javase/9/docs/api/java/util/zip/CRC32C.html > Platform specific assembler intrinsics make it more effective than any pure > Java implementation. > Hadoop is compiled against Java 8 but class constructor may be accessible > with method handle on 9 to instances implementing Checksum in runtime. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15033) Use java.util.zip.CRC32C for Java 9 and above
[ https://issues.apache.org/jira/browse/HADOOP-15033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Chuyko updated HADOOP-15033: --- Attachment: HADOOP-15033.001.patch > Use java.util.zip.CRC32C for Java 9 and above > - > > Key: HADOOP-15033 > URL: https://issues.apache.org/jira/browse/HADOOP-15033 > Project: Hadoop Common > Issue Type: Improvement > Components: performance, util >Affects Versions: 3.0.0 >Reporter: Dmitry Chuyko > Attachments: HADOOP-15033.001.patch > > > java.util.zip.CRC32C implementation is available since Java 9. > https://docs.oracle.com/javase/9/docs/api/java/util/zip/CRC32C.html > Platform specific assembler intrinsics make it more effective than any pure > Java implementation. > Hadoop is compiled against Java 8 but class constructor may be accessible > with method handle on 9 to instances implementing Checksum in runtime. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15033) Use java.util.zip.CRC32C for Java 9 and above
[ https://issues.apache.org/jira/browse/HADOOP-15033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256722#comment-16256722 ] Dmitry Chuyko commented on HADOOP-15033: Results on latest JDK 10 on 8 core ARM64 including native library: aarch64, 8 cores, JDK 10 java.version = 10-internal java.runtime.name = OpenJDK Runtime Environment java.runtime.version = 10-internal+0-adhoc.root.hs10 java.vm.version = 10-internal+0-adhoc.root.hs10 java.vm.vendor = Oracle Corporation java.vm.name = OpenJDK 64-Bit Server VM java.vm.specification.version = 10 java.specification.version = 10 os.arch = aarch64 os.name = Linux os.version = 4.9.23-std-1 Data Length = 64 MB Trials = 5 Direct Buffer Performance Table (bpc: byte-per-crc in MB/sec; #T: #Theads) | bpc | #T || Zip || ZipC | % diff || PureJava | % diff || PureJavaC | % diff || Native | % diff || NativeC | % diff | |32 | 1 | 168.5 | 182.7 | 8.4% | 101.3 | -44.6% | 93.5 | -7.7% | 455.8 | 387.3% | 465.6 | 2.1% | |32 | 2 | 173.3 | 179.7 | 3.7% | 101.6 | -43.5% | 96.4 | -5.1% | 396.0 | 310.9% | 421.8 | 6.5% | |32 | 4 | 165.3 | 164.9 | -0.3% | 98.3 | -40.4% | 92.8 | -5.6% | 393.3 | 323.7% | 389.9 | -0.9% | |32 | 8 | 164.3 | 163.4 | -0.5% | 98.3 | -39.9% | 92.5 | -5.9% | 395.6 | 327.8% | 391.9 | -0.9% | |32 | 16 | 87.9 | 87.5 | -0.4% | 50.8 | -42.0% | 48.3 | -4.9% | 230.6 | 377.7% | 238.1 | 3.2% | | bpc | #T || Zip || ZipC | % diff || PureJava | % diff || PureJavaC | % diff || Native | % diff || NativeC | % diff | |64 | 1 | 253.6 | 253.7 | 0.0% | 121.9 | -52.0% | 116.5 | -4.4% | 460.3 | 295.2% | 458.3 | -0.4% | |64 | 2 | 238.9 | 237.9 | -0.4% | 119.5 | -49.8% | 115.6 | -3.2% | 440.2 | 280.6% | 437.8 | -0.5% | |64 | 4 | 239.2 | 239.2 | 0.0% | 119.8 | -49.9% | 115.6 | -3.5% | 438.3 | 279.1% | 477.8 | 9.0% | |64 | 8 | 237.2 | 236.0 | -0.5% | 116.3 | -50.7% | 115.0 | -1.1% | 458.1 | 298.3% | 440.1 | -3.9% | |64 | 16 | 129.0 | 128.4 | -0.4% | 62.2 | -51.6% | 60.2 | -3.2% | 261.5 | 334.3% | 257.5 | -1.5% | | bpc | #T || Zip || ZipC | % diff || PureJava | % diff || PureJavaC | % diff || Native | % diff || NativeC | % diff | | 128 | 1 | 604.0 | 616.3 | 2.0% | 179.8 | -70.8% | 181.1 | 0.8% | 480.6 | 165.3% | 484.3 | 0.8% | | 128 | 2 | 606.7 | 608.8 | 0.4% | 180.1 | -70.4% | 180.7 | 0.4% | 462.4 | 155.9% | 461.2 | -0.3% | | 128 | 4 | 608.9 | 599.3 | -1.6% | 180.3 | -69.9% | 180.3 | 0.0% | 461.4 | 155.8% | 462.6 | 0.3% | | 128 | 8 | 596.1 | 602.2 | 1.0% | 178.4 | -70.4% | 179.5 | 0.6% | 466.3 | 159.8% | 462.2 | -0.9% | | 128 | 16 | 357.8 | 349.6 | -2.3% | 92.9 | -73.4% | 93.5 | 0.6% | 277.4 | 196.7% | 284.6 | 2.6% | | bpc | #T || Zip || ZipC | % diff || PureJava | % diff || PureJavaC | % diff || Native | % diff || NativeC | % diff | | 256 | 1 | 793.2 | 794.3 | 0.1% | 200.7 | -74.7% | 209.0 | 4.1% | 533.3 | 155.2% | 485.6 | -8.9% | | 256 | 2 | 802.6 | 770.6 | -4.0% | 199.0 | -74.2% | 205.3 | 3.1% | 645.8 | 214.6% | 576.9 | -10.7% | | 256 | 4 | 844.8 | 802.7 | -5.0% | 199.7 | -75.1% | 195.4 | -2.1% | 556.0 | 184.5% | 513.9 | -7.6% | | 256 | 8 | 778.8 | 760.2 | -2.4% | 195.4 | -74.3% | 202.6 | 3.7% | 503.7 | 148.7% | 483.8 | -4.0% | | 256 | 16 | 434.1 | 459.8 | 5.9% | 106.1 | -76.9% | 107.9 | 1.7% | 287.8 | 166.7% | 279.6 | -2.8% | | bpc | #T || Zip || ZipC | % diff || PureJava | % diff || PureJavaC | % diff || Native | % diff || NativeC | % diff | | 512 | 1 | 796.8 | 805.1 | 1.0% | 199.8 | -75.2% | 209.6 | 4.9% | 506.7 | 141.7% | 566.2 | 11.7% | | 512 | 2 | 777.7 | 883.4 | 13.6% | 202.1 | -77.1% | 211.9 | 4.9% | 651.2 | 207.3% | 653.5 | 0.4% | | 512 | 4 | 849.1 | 858.6 | 1.1% | 198.2 | -76.9% | 212.2 | 7.1% | 618.4 | 191.4% | 558.3 | -9.7% | | 512 | 8 | 793.7 | 813.7 | 2.5% | 195.1 | -76.0% | 208.2 | 6.7% | 539.7 | 159.2% | 537.8 | -0.4% | | 512 | 16 | 506.5 | 473.8 | -6.5% | 107.5 | -77.3% | 106.4 | -1.0% | 305.6 | 187.0% | 282.5 | -7.6% | |
[jira] [Commented] (HADOOP-15033) Use java.util.zip.CRC32C for Java 9 and above
[ https://issues.apache.org/jira/browse/HADOOP-15033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256717#comment-16256717 ] Dmitry Chuyko commented on HADOOP-15033: Results on latest JDK 10 on 4 core x86 including native library: java.version = 10-internal java.runtime.name = OpenJDK Runtime Environment java.runtime.version = 10-internal+0-adhoc.tp.hs10 java.vm.version = 10-internal+0-adhoc.tp.hs10 java.vm.vendor = Oracle Corporation java.vm.name = OpenJDK 64-Bit Server VM java.vm.specification.version = 10 java.specification.version = 10 os.arch = amd64 os.name = Linux os.version = 4.13.0-16-generic Data Length = 64 MB Trials = 5 Direct Buffer Performance Table (bpc: byte-per-crc in MB/sec; #T: #Theads) | bpc | #T || Zip || ZipC | % diff || PureJava | % diff || PureJavaC | % diff || Native | % diff || NativeC | % diff | |32 | 1 | 540.5 | 921.7 | 70.5% | 644.0 | -30.1% | 662.5 | 2.9% |1535.3 | 131.7% |8598.1 | 460.0% | |32 | 2 | 346.0 | 489.6 | 41.5% | 325.3 | -33.6% | 303.2 | -6.8% | 898.0 | 196.2% |4720.6 | 425.7% | |32 | 4 | 226.8 | 384.1 | 69.3% | 238.6 | -37.9% | 242.5 | 1.6% | 731.9 | 201.9% |3838.0 | 424.4% | |32 | 8 | 104.1 | 213.6 | 105.1% | 147.6 | -30.9% | 141.9 | -3.9% | 386.5 | 172.4% |2283.9 | 490.9% | |32 | 16 | 70.0 | 110.1 | 57.1% | 77.6 | -29.5% | 74.9 | -3.5% | 223.6 | 198.6% |1657.4 | 641.1% | | bpc | #T || Zip || ZipC | % diff || PureJava | % diff || PureJavaC | % diff || Native | % diff || NativeC | % diff | |64 | 1 | 828.0 |1266.5 | 53.0% | 876.1 | -30.8% | 833.9 | -4.8% |1592.7 | 91.0% | 10217.4 | 541.5% | |64 | 2 | 549.2 | 753.1 | 37.1% | 420.2 | -44.2% | 438.2 | 4.3% |1070.3 | 144.3% |6534.0 | 510.5% | |64 | 4 | 384.0 | 627.6 | 63.5% | 348.8 | -44.4% | 365.9 | 4.9% | 854.0 | 133.4% |4762.2 | 457.6% | |64 | 8 | 226.6 | 317.2 | 40.0% | 195.5 | -38.4% | 218.6 | 11.8% | 488.6 | 123.5% |2760.0 | 464.9% | |64 | 16 | 113.7 | 176.1 | 54.9% | 113.8 | -35.4% | 125.4 | 10.2% | 255.9 | 104.1% |2556.6 | 899.1% | | bpc | #T || Zip || ZipC | % diff || PureJava | % diff || PureJavaC | % diff || Native | % diff || NativeC | % diff | | 128 | 1 |1146.1 |2241.7 | 95.6% |1080.8 | -51.8% | 1008.1 | -6.7% |1626.2 | 61.3% | 11525.3 | 608.7% | | 128 | 2 | 792.6 |1248.3 | 57.5% | 536.0 | -57.1% | 786.1 | 46.7% |1360.0 | 73.0% |7295.6 | 436.5% | | 128 | 4 | 602.7 |1074.4 | 78.3% | 456.8 | -57.5% | 437.4 | -4.2% | 927.7 | 112.1% |6405.6 | 590.5% | | 128 | 8 | 327.6 | 607.0 | 85.3% | 265.6 | -56.2% | 245.3 | -7.6% | 499.6 | 103.7% |3545.9 | 609.8% | | 128 | 16 | 176.2 | 295.0 | 67.5% | 150.7 | -48.9% | 132.4 | -12.1% | 287.0 | 116.7% |2865.1 | 898.3% | | bpc | #T || Zip || ZipC | % diff || PureJava | % diff || PureJavaC | % diff || Native | % diff || NativeC | % diff | | 256 | 1 |1647.9 |2802.3 | 70.0% |1240.3 | -55.7% | 1090.2 | -12.1% |1525.0 | 39.9% | 11362.2 | 645.0% | | 256 | 2 |1021.1 |1621.4 | 58.8% | 642.2 | -60.4% | 616.8 | -4.0% |1241.3 | 101.3% |7501.2 | 504.3% | | 256 | 4 | 879.3 |1368.5 | 55.6% | 494.9 | -63.8% | 497.2 | 0.5% |1001.9 | 101.5% |5424.7 | 441.4% | | 256 | 8 | 448.7 | 705.8 | 57.3% | 302.1 | -57.2% | 266.1 | -11.9% | 529.9 | 99.1% |4327.7 | 716.7% | | 256 | 16 | 245.4 | 477.3 | 94.5% | 147.3 | -69.1% | 152.9 | 3.8% | 325.5 | 113.0% |2456.4 | 654.6% | | bpc | #T || Zip || ZipC | % diff || PureJava | % diff || PureJavaC | % diff || Native | % diff || NativeC | % diff | | 512 | 1 |1872.1 |2954.9 | 57.8% |1307.1 | -55.8% | 1102.0 | -15.7% |1586.3 | 44.0% |9733.8 | 513.6% | | 512 | 2 |1237.1 |1940.5 | 56.9% | 707.8 | -63.5% | 664.8 | -6.1% |1278.6 | 92.3% |7194.6 | 462.7% | | 512 | 4 | 997.9 |1619.2 | 62.3% | 610.2 | -62.3% | 553.2 | -9.4% |1035.6 | 87.2% |5029.4 | 385.7% | | 512 | 8 | 554.1 | 974.3 | 75.8% | 329.6 | -66.2% | 299.1 | -9.3% | 596.8 | 99.5% |4345.3 | 628.1% | | 512 | 16 | 327.0 | 578.3 | 76.9% | 186.9 | -67.7% | 178.8 | -4.4% | 336.4 | 88.2% |3274.8 | 873.4% | | bpc | #T || Zip ||
[jira] [Comment Edited] (HADOOP-15033) Use java.util.zip.CRC32C for Java 9 and above
[ https://issues.apache.org/jira/browse/HADOOP-15033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16251631#comment-16251631 ] Dmitry Chuyko edited comment on HADOOP-15033 at 11/17/17 9:24 AM: -- It worths mentioning here that java.util.zip pure Java implementation compiled by C2 without some inlining is ~2x slower for len=512 than Hadoop's pure Java though they basically use the same approach to calculation. It is a subject for improvement in core library for platforms and JVMS that don't provide CRC intrinsics. But again in real life on Hotspot on x86 and aarch64 you always get an intrinsic implementation. was (Author: dchuyko): It worths mentioning here that java.util.zip pure Java implementation compiled by C2 is ~2x slower for len=512 than Hadoop's pure Java though they basically use the same approach to calculation. It is a subject for improvement in core library for platforms and JVMS that don't provide CRC intrinsics. But again in real life on Hotspot on x86 and aarch64 you always get an intrinsic implementation. > Use java.util.zip.CRC32C for Java 9 and above > - > > Key: HADOOP-15033 > URL: https://issues.apache.org/jira/browse/HADOOP-15033 > Project: Hadoop Common > Issue Type: Improvement > Components: performance, util >Affects Versions: 3.0.0 >Reporter: Dmitry Chuyko > > java.util.zip.CRC32C implementation is available since Java 9. > https://docs.oracle.com/javase/9/docs/api/java/util/zip/CRC32C.html > Platform specific assembler intrinsics make it more effective than any pure > Java implementation. > Hadoop is compiled against Java 8 but class constructor may be accessible > with method handle on 9 to instances implementing Checksum in runtime. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15033) Use java.util.zip.CRC32C for Java 9 and above
[ https://issues.apache.org/jira/browse/HADOOP-15033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16251631#comment-16251631 ] Dmitry Chuyko commented on HADOOP-15033: It worths mentioning here that java.util.zip pure Java implementation compiled by C2 is ~2x slower for len=512 than Hadoop's pure Java though they basically use the same approach to calculation. It is a subject for improvement in core library for platforms and JVMS that don't provide CRC intrinsics. But again in real life on Hotspot on x86 and aarch64 you always get an intrinsic implementation. > Use java.util.zip.CRC32C for Java 9 and above > - > > Key: HADOOP-15033 > URL: https://issues.apache.org/jira/browse/HADOOP-15033 > Project: Hadoop Common > Issue Type: Improvement > Components: performance, util >Affects Versions: 3.0.0 >Reporter: Dmitry Chuyko > > java.util.zip.CRC32C implementation is available since Java 9. > https://docs.oracle.com/javase/9/docs/api/java/util/zip/CRC32C.html > Platform specific assembler intrinsics make it more effective than any pure > Java implementation. > Hadoop is compiled against Java 8 but class constructor may be accessible > with method handle on 9 to instances implementing Checksum in runtime. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-15033) Use java.util.zip.CRC32C for Java 9 and above
[ https://issues.apache.org/jira/browse/HADOOP-15033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16251538#comment-16251538 ] Dmitry Chuyko edited comment on HADOOP-15033 at 11/14/17 3:38 PM: -- [~raviprak] I compared partial checksums over random array and byte buffer. Some more details: * Over 1000 elements (crc of 0 element, crc of 1 element, crc of 2 elements...). * Zero and non-zero offset. * Interpreter, C1, C2 Hotspot compiler. * OpenJDK 10-internal * x86_64 and aarch64 CRC32C values required to be equal are equal. was (Author: dchuyko): [~raviprak] I compared partial checksums over partial random array and byte buffer. Some more details: * Over 1000 elements (crc of 0 element, crc of 1 element...). * Zero and non-zero offset. * Interpreter, C1, C2 Hotspot compiler. * OpenJDK 10-internal * x86_64 and aarch64 CRC32C values required to be equal are equal. > Use java.util.zip.CRC32C for Java 9 and above > - > > Key: HADOOP-15033 > URL: https://issues.apache.org/jira/browse/HADOOP-15033 > Project: Hadoop Common > Issue Type: Improvement > Components: performance, util >Affects Versions: 3.0.0 >Reporter: Dmitry Chuyko > > java.util.zip.CRC32C implementation is available since Java 9. > https://docs.oracle.com/javase/9/docs/api/java/util/zip/CRC32C.html > Platform specific assembler intrinsics make it more effective than any pure > Java implementation. > Hadoop is compiled against Java 8 but class constructor may be accessible > with method handle on 9 to instances implementing Checksum in runtime. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15033) Use java.util.zip.CRC32C for Java 9 and above
[ https://issues.apache.org/jira/browse/HADOOP-15033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16251538#comment-16251538 ] Dmitry Chuyko commented on HADOOP-15033: [~raviprak] I compared partial checksums over partial random array and byte buffer. Some more details: * Over 1000 elements (crc of 0 element, crc of 1 element...). * Zero and non-zero offset. * Interpreter, C1, C2 Hotspot compiler. * OpenJDK 10-internal * x86_64 and aarch64 CRC32C values required to be equal are equal. > Use java.util.zip.CRC32C for Java 9 and above > - > > Key: HADOOP-15033 > URL: https://issues.apache.org/jira/browse/HADOOP-15033 > Project: Hadoop Common > Issue Type: Improvement > Components: performance, util >Affects Versions: 3.0.0 >Reporter: Dmitry Chuyko > > java.util.zip.CRC32C implementation is available since Java 9. > https://docs.oracle.com/javase/9/docs/api/java/util/zip/CRC32C.html > Platform specific assembler intrinsics make it more effective than any pure > Java implementation. > Hadoop is compiled against Java 8 but class constructor may be accessible > with method handle on 9 to instances implementing Checksum in runtime. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15033) Use java.util.zip.CRC32C for Java 9 and above
[ https://issues.apache.org/jira/browse/HADOOP-15033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Chuyko updated HADOOP-15033: --- Description: java.util.zip.CRC32C implementation is available since Java 9. https://docs.oracle.com/javase/9/docs/api/java/util/zip/CRC32C.html Platform specific assembler intrinsics make it more effective than any pure Java implementation. Hadoop is compiled against Java 8 but class constructor may be accessible with method handle on 9 to instances implementing Checksum in runtime. was: java.util.zip.CRC32C implementation is available since Java 9. https://docs.oracle.com/javase/9/docs/api/java/util/zip/CRC32C.html Platform specific assembler intrinsics make it more aeefective than any pure Java implementation. Hadoop is compiled against Java 8 but class constructor may be accessible with method handle on 9 to instances implementing Checksum in runtime. > Use java.util.zip.CRC32C for Java 9 and above > - > > Key: HADOOP-15033 > URL: https://issues.apache.org/jira/browse/HADOOP-15033 > Project: Hadoop Common > Issue Type: Improvement > Components: performance, util >Reporter: Dmitry Chuyko > > java.util.zip.CRC32C implementation is available since Java 9. > https://docs.oracle.com/javase/9/docs/api/java/util/zip/CRC32C.html > Platform specific assembler intrinsics make it more effective than any pure > Java implementation. > Hadoop is compiled against Java 8 but class constructor may be accessible > with method handle on 9 to instances implementing Checksum in runtime. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15033) Use java.util.zip.CRC32C for Java 9 and above
[ https://issues.apache.org/jira/browse/HADOOP-15033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16249379#comment-16249379 ] Dmitry Chuyko commented on HADOOP-15033: In the results above mainline JDK 10 was used. > Use java.util.zip.CRC32C for Java 9 and above > - > > Key: HADOOP-15033 > URL: https://issues.apache.org/jira/browse/HADOOP-15033 > Project: Hadoop Common > Issue Type: Improvement > Components: performance, util >Reporter: Dmitry Chuyko > > java.util.zip.CRC32C implementation is available since Java 9. > https://docs.oracle.com/javase/9/docs/api/java/util/zip/CRC32C.html > Platform specific assembler intrinsics make it more aeefective than any pure > Java implementation. > Hadoop is compiled against Java 8 but class constructor may be accessible > with method handle on 9 to instances implementing Checksum in runtime. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15033) Use java.util.zip.CRC32C for Java 9 and above
[ https://issues.apache.org/jira/browse/HADOOP-15033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16249316#comment-16249316 ] Dmitry Chuyko commented on HADOOP-15033: Related issue https://issues.apache.org/jira/browse/HADOOP-10674 > Use java.util.zip.CRC32C for Java 9 and above > - > > Key: HADOOP-15033 > URL: https://issues.apache.org/jira/browse/HADOOP-15033 > Project: Hadoop Common > Issue Type: Improvement > Components: performance, util >Reporter: Dmitry Chuyko > > java.util.zip.CRC32C implementation is available since Java 9. > https://docs.oracle.com/javase/9/docs/api/java/util/zip/CRC32C.html > Platform specific assembler intrinsics make it more aeefective than any pure > Java implementation. > Hadoop is compiled against Java 8 but class constructor may be accessible > with method handle on 9 to instances implementing Checksum in runtime. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-15033) Use java.util.zip.CRC32C for Java 9 and above
Dmitry Chuyko created HADOOP-15033: -- Summary: Use java.util.zip.CRC32C for Java 9 and above Key: HADOOP-15033 URL: https://issues.apache.org/jira/browse/HADOOP-15033 Project: Hadoop Common Issue Type: Improvement Components: performance, util Reporter: Dmitry Chuyko java.util.zip.CRC32C implementation is available since Java 9. https://docs.oracle.com/javase/9/docs/api/java/util/zip/CRC32C.html Platform specific assembler intrinsics make it more aeefective than any pure Java implementation. Hadoop is compiled against Java 8 but class constructor may be accessible with method handle on 9 to instances implementing Checksum in runtime. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13363) Upgrade protobuf from 2.5.0 to something newer
[ https://issues.apache.org/jira/browse/HADOOP-13363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246154#comment-16246154 ] Dmitry Chuyko commented on HADOOP-13363: Current usage scheme leads to a small tricky problem. Consider available versions: apt-cache showpkg protobuf-compiler '2.6.1-1.3' only on Ununtu 16.04 (LTS) aarch64 '3.0.0-9ubuntu5' only on Ubuntu 17.10 amd64 Probably both versions will work when manually installed. Also if I understand it right build system docker image will need an update. Not sure about other distros and release schedules but it looks right to stick to a current version from some LTS distro of next such version. For next LTS Ubuntu it is currently also 3.0.0: '3.0.0-9ubuntu5' https://packages.ubuntu.com/search?keywords=protobuf-compiler > Upgrade protobuf from 2.5.0 to something newer > -- > > Key: HADOOP-13363 > URL: https://issues.apache.org/jira/browse/HADOOP-13363 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.0.0-alpha1, 3.0.0-alpha2 >Reporter: Allen Wittenauer >Assignee: Tsuyoshi Ozawa > Attachments: HADOOP-13363.001.patch, HADOOP-13363.002.patch, > HADOOP-13363.003.patch, HADOOP-13363.004.patch, HADOOP-13363.005.patch > > > Standard protobuf 2.5.0 does not work properly on many platforms. (See, for > example, https://gist.github.com/BennettSmith/7111094 ). In order for us to > avoid crazy work arounds in the build environment and the fact that 2.5.0 is > starting to slowly disappear as a standard install-able package for even > Linux/x86, we need to either upgrade or self bundle or something else. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13363) Upgrade protobuf from 2.5.0 to something newer
[ https://issues.apache.org/jira/browse/HADOOP-13363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245595#comment-16245595 ] Dmitry Chuyko commented on HADOOP-13363: [~andrew.wang] On some platforms version 2.5.0 is not even buildable, e.g. on AArch64. While for example version 2.6.1 is available in Linux distribution repositories and seems to be ok. > Upgrade protobuf from 2.5.0 to something newer > -- > > Key: HADOOP-13363 > URL: https://issues.apache.org/jira/browse/HADOOP-13363 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.0.0-alpha1, 3.0.0-alpha2 >Reporter: Allen Wittenauer >Assignee: Tsuyoshi Ozawa > Attachments: HADOOP-13363.001.patch, HADOOP-13363.002.patch, > HADOOP-13363.003.patch, HADOOP-13363.004.patch, HADOOP-13363.005.patch > > > Standard protobuf 2.5.0 does not work properly on many platforms. (See, for > example, https://gist.github.com/BennettSmith/7111094 ). In order for us to > avoid crazy work arounds in the build environment and the fact that 2.5.0 is > starting to slowly disappear as a standard install-able package for even > Linux/x86, we need to either upgrade or self bundle or something else. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
