Sure, I have done that. https://issues.apache.org/jira/browse/PHOENIX-2469
2015-11-30 22:22 GMT+01:00 Josh Mahonin <[email protected]>: > Hi David, > > Thanks for the bug report and the proposed patch. Please file a JIRA and > we'll take the discussion there. > > Josh > > On Mon, Nov 30, 2015 at 1:01 PM, Dawid Wysakowicz < > [email protected]> wrote: > >> Hi, >> >> I've recently found some behaviour that I found buggy when working with >> phoenix-spark and arrays. >> >> Take a look at those unit tests: >> >> test("Can save arrays from custom dataframes back to phoenix") { >> val dataSet = List(Row(2L, Array("String1", "String2", "String3"))) >> >> val sqlContext = new SQLContext(sc) >> >> val schema = StructType( >> Seq(StructField("ID", LongType, nullable = false), >> StructField("VCARRAY", ArrayType(StringType)))) >> >> val rowRDD = sc.parallelize(dataSet) >> >> // Apply the schema to the RDD. >> val df = sqlContext.createDataFrame(rowRDD, schema) >> >> df.write >> .format("org.apache.phoenix.spark") >> .options(Map("table" -> "ARRAY_TEST_TABLE", "zkUrl" -> >> quorumAddress)) >> .mode(SaveMode.Overwrite) >> .save() >> } >> >> test("Can save arrays of AnyVal type back to phoenix") { >> val dataSet = List((2L, Array(1, 2, 3), Array(1L, 2L, 3L))) >> >> sc >> .parallelize(dataSet) >> .saveToPhoenix( >> "ARRAY_ANYVAL_TEST_TABLE", >> Seq("ID", "INTARRAY", "BIGINTARRAY"), >> zkUrl = Some(quorumAddress) >> ) >> >> // Load the results back >> val stmt = conn.createStatement() >> val rs = stmt.executeQuery("SELECT INTARRAY, BIGINTARRAY FROM >> ARRAY_ANYVAL_TEST_TABLE WHERE ID = 2") >> rs.next() >> val intArray = rs.getArray(1).getArray().asInstanceOf[Array[Int]] >> val longArray = rs.getArray(2).getArray().asInstanceOf[Array[Long]] >> >> // Verify the arrays are equal >> intArray shouldEqual dataSet(0)._2 >> longArray shouldEqual dataSet(0)._3 >> } >> >> Both fail with some ClassCastExceptions. >> >> In attached patch I've proposed a solution. The tricky part is with >> Array[Byte] as this would be same for both VARBINARY and TINYINT[]. >> >> Let me know If I should create an issue for this, and if my solution >> satisfies you. >> >> Regards >> Dawid Wysakowicz >> >> >> >
