[jira] [Commented] (BEAM-225) Create Class for Common TypeDescriptors

[Jesse Anderson (JIRA)]

    [ 
https://issues.apache.org/jira/browse/BEAM-225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15269278#comment-15269278
 ] 

Jesse Anderson commented on BEAM-225:
-------------------------------------

I used a Python script to generate this code. I think it will be helpful to 
others with many different custom TypeDescriptors. You can add your custom 
classes, remove the other primitive classes, and change the class names.

{code}
#!/usr/bin/python

import sys, re, os

if len(sys.argv) != 2:
  raise ValueError("Usage: outputdir")

outputDirStr = sys.argv[1]

if os.path.exists(outputDirStr):
  if os.path.isdir(outputDirStr) == False:
    raise ValueError("outputdir must be a directory")
else:
  os.makedirs(outputDirStr)

print "Output Dir:" + outputDirStr

primitiveTypes = 
["Boolean","Double","Float","Integer","Long","Short","BigDecimal","String"]

regularMethod =\
"""
  /**
   * The {{@link TypeDescriptor}} for {0}.
   * This is the equivalent of:
   * <pre>
   * new TypeDescriptor&lt;{0}&gt;() {{}};
   * </pre>
   * @return A {{@link TypeDescriptor}} for {0}
   */
  public static TypeDescriptor<{0}> {1}() {{
    return new TypeDescriptor<{0}>() {{}};
  }}
"""

kvMethod =\
"""
  /**
   * The {{@link TypeDescriptor}} for {{@link KV}} of type {0} and {2}.
   * This is the equivalent of:
   * <pre>
   * new TypeDescriptor&lt;KV&lt;{0},{2}&gt;&gt;() {{}};
   * </pre>
   * @return A {{@link TypeDescriptor}} for {{@link KV}} of type {0} and {2}
   */
  public static TypeDescriptor<KV<{0}, {2}>>
    {1}{3}() {{
    return new TypeDescriptor<KV<{0}, {2}>>() {{}};
  }}
"""

### Write out Type Descriptors

typeDescriptorsHeader =\
"""package TODO;

import java.math.BigDecimal;

/**
 * TODO
 */
public class TypeDescriptors {"""

outputFile = open(outputDirStr + "/TypeDescriptors.java",'w')
outputFile.write(typeDescriptorsHeader)

for primitiveType in primitiveTypes:
  outputFile.write(regularMethod.format(primitiveType, (primitiveType + 
"s").lower()))

outputFile.write("}\n")
outputFile.close()

### Write out KV Type Descriptors

kvTypeDescriptorsHeader =\
"""package TODO;

import java.math.BigDecimal;

/**
 * TODO
 */
public class KVTypeDescriptors {"""

outputFile = open(outputDirStr + "/KVTypeDescriptors.java",'w')
outputFile.write(kvTypeDescriptorsHeader)

for primitiveType in primitiveTypes:
  for valuePrimitiveType in primitiveTypes:
    outputFile.write(kvMethod.format(primitiveType, (primitiveType + 
"s").lower(), valuePrimitiveType, valuePrimitiveType + "s"))

outputFile.write("}\n")
outputFile.close()
{code}

> Create Class for Common TypeDescriptors
> ---------------------------------------
>
>                 Key: BEAM-225
>                 URL: https://issues.apache.org/jira/browse/BEAM-225
>             Project: Beam
>          Issue Type: Bug
>          Components: sdk-java-core
>            Reporter: Jesse Anderson
>            Priority: Trivial
>              Labels: starter
>
> There should be a built-in class for common types like String, Float, etc.
> Right now, all types have to create an inline TypeDescriptor:
> {code:java}
>         PCollection<String> words = suits.apply(
>                 FlatMapElements.via(
>                         (String line) -> Arrays.asList(line.split(" "))
>                 ).withOutputType(new TypeDescriptor<String>() {}));
> {code}
> The should be a built-in class with common types like String so you don't 
> have to create a TypeDescriptor each time like:
> {code:java}
>               PCollection<String> words = suits.apply(
>                 FlatMapElements.via(
>                         (String line) -> Arrays.asList(line.split(" "))
>                 ).withOutputType(TypeDescriptors.STRINGS));
> {code}
> Another possibility is to make it a static method:
> {code:java}
>               PCollection<String> words = suits.apply(
>                 FlatMapElements.via(
>                         (String line) -> Arrays.asList(line.split(" "))
>                 ).withOutputType(TypeDescriptors.strings()));
> {code}
> An example of this is Apache Crunch's Writables class 
> https://crunch.apache.org/apidocs/0.11.0/org/apache/crunch/types/writable/Writables.html.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)