Hey Dave, Which file did you add the "forceBits()" method to ?
On Fri, Feb 5, 2016 at 8:33 AM, Dave Oshinsky <[email protected]> wrote: > Hi Drill-ers, > I am looking into fixing JIRA > https://issues.apache.org/jira/browse/DRILL-4184. I've encountered a > number of strange build problems along the way with my drill 1.4 snapshot, > including inability to rebuild after running "mvn clean", no matter what I > try. So, I'm building from scratch for the second time, at least. The > latest problem really has me stumped at the moment. I added a > "forceBits(int,int)" method that I see in generated source file > NullableDecimal28SparseVector.java (and > NullableDecimal38SparseVector.java), but somehow this doesn't get compiled > properly into the *.class and my build keeps failing as if the new > forceBits method isn't there: > > [ERROR] Failed to execute goal > org.apache.maven.plugins:maven-compiler-plugin:3.2:compile > (default-compile) on project drill-java-exec: Compilation failure: > Compilation failure: > [ERROR] > C:\apache\apache-drill-1.4.0\rebuild2\drill-1.4.0\exec\java-exec\src\main\java\org\apache\drill\exec\store\parquet\columnreaders\VarLengthColumnReaders.java:[108,29] > error: cannot find symbol > [ERROR] symbol: method forceBits(int,int) > [ERROR] location: variable nullableDecimal28Vector of type > NullableDecimal28SparseVector > [ERROR] > C:\apache\apache-drill-1.4.0\rebuild2\drill-1.4.0\exec\java-exec\src\main\java\org\apache\drill\exec\store\parquet\columnreaders\VarLengthColumnReaders.java:[179,29] > error: cannot find symbol > > Can anyone suggest how to fix this without starting over from scratch in a > new build node (again)? Any advice would be greatly appreciated. > > I will send a separate email eventually regarding the design of my fix, > which I know is only a short-term solution to the problem of handling > variable width decimal fields in Parquet files. To make a long story > short, all the decimal vectors are fixed width vectors, which don't have > the ability to "remember" varying sizes from one decimal field to the > next. I've hacked up something to "remember" the varying field sizes > (BigDecimal array sizes) in NullableVarLengthValuesColumn and > VarLengthValuesColumn, not in the decimal vectors. This seems to work, > though it's admittedly ugly. However, I ran into a problem with nullable > varying width decimal columns where the "isSet" always returns 0, as if the > column is null, when it is not, and the sparse decimal data is present in > the vector (but Drill won't send the decimal value, because it thinks it's > null). Hence the "forceBits" hack to try to work around this. It seemed > like I was close to running a successful Drill query on the varying width > decimal Parquet data, but alas, I ran into (another) build problem. I do > have a LOT of questions as to why the decimal stuff was designed the way it > is, but that's for another email.... > > Thanks, > Dave Oshinsky > > > > > > ***************************Legal Disclaimer*************************** > "This communication may contain confidential and privileged material for > the > sole use of the intended recipient. Any unauthorized review, use or > distribution > by others is strictly prohibited. If you have received the message by > mistake, > please advise the sender by reply email and delete the message. Thank you." > ********************************************************************** -- Abdelhakim Deneche Software Engineer <http://www.mapr.com/> Now Available - Free Hadoop On-Demand Training <http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available>
