Actually, another way to make it work is removing the blank line. Below little program shows that readtable() accepts test_df1 and test_df2, but fails on test_df3.
Also, the fact that it started to happen today had nothing todo with Julia or DataFrame updates. The file is created by Stan and the latest version inserts that blank line. Of course I could clean up the file, but maybe this is an issue in DataFrame's readtable function? Apologies for the earlier incomplete report. Rob J. Goedman [email protected] julia> include("/Users/rob/.julia/v0.3/MCMCExampleRepository/test/test_df.jl") 4x10 DataFrame |-------|---------------|---------|---------| | Col # | Name | Eltype | Missing | | 1 | lp__ | Float64 | 0 | | 2 | accept_stat__ | Float64 | 0 | | 3 | stepsize__ | Float64 | 0 | | 4 | treedepth__ | Int64 | 0 | | 5 | n_leapfrog__ | Int64 | 0 | | 6 | n_divergent__ | Int64 | 0 | | 7 | beta_1 | Float64 | 0 | | 8 | beta_2 | Float64 | 0 | | 9 | beta_3 | Float64 | 0 | | 10 | sigma | Float64 | 0 | 4x10 DataFrame |-------|---------------|---------|---------| | Col # | Name | Eltype | Missing | | 1 | lp__ | Float64 | 0 | | 2 | accept_stat__ | Float64 | 0 | | 3 | stepsize__ | Float64 | 0 | | 4 | treedepth__ | Int64 | 0 | | 5 | n_leapfrog__ | Int64 | 0 | | 6 | n_divergent__ | Int64 | 0 | | 7 | beta_1 | Float64 | 0 | | 8 | beta_2 | Float64 | 0 | | 9 | beta_3 | Float64 | 0 | | 10 | sigma | Float64 | 0 | ERROR: BoundsError() in findcorruption at /Users/rob/.julia/v0.3/DataFrames/src/dataframe/io.jl:663 in readtable! at /Users/rob/.julia/v0.3/DataFrames/src/dataframe/io.jl:731 in readtable at /Users/rob/.julia/v0.3/DataFrames/src/dataframe/io.jl:812 in readtable at /Users/rob/.julia/v0.3/DataFrames/src/dataframe/io.jl:879 in include at boot.jl:244 while loading /Users/rob/.julia/v0.3/MCMCExampleRepository/test/test_df.jl, in expression starting on line 11 julia> On May 20, 2014, at 6:36 PM, Rob J. Goedman <[email protected]> wrote: > Hi, > > Using a freshly updated Version 0.3.0-prerelease+3251 (2014-05-20 23:18 UTC) > of Julia I think I noticed a different behavior of readtable(), which I hope > is not intended. > > I have a small test file with data as shown below (and attached as a file at > the end of the email): > > lp__,accept_stat__,stepsize__,treedepth__,n_leapfrog__,n_divergent__,mu > # Adaptation terminated > > -19.8871,0.975123,0.303529,4,15,0,4.25051 > -22.1208,0.971631,0.303529,3,7,0,8.55276 > -23.8336,0.857954,0.303529,4,15,0,4.41087 > > If I remove the commented line ("# Adaptation terminated"), readtable() has > no problem, but if it's there readtable() seems to ignore the > 'allowcomments=true'. > > I didn't update DataFrames as far as I am aware, but once or twice today I > did pull Julia's master from github. > > I wonder if someone could try this example. Thanks a lot. > > Rob J. Goedman > [email protected] > > > julia> df = readtable("schools8_samples.csv", allowcomments=true) > ERROR: Saw 4 rows, 5 columns and 22 fields > * Line 1 has 3 columns > > in error at error.jl:21 > in findcorruption at > /Users/rob/.julia/v0.3/DataFrames/src/dataframe/io.jl:680 > in readtable! at /Users/rob/.julia/v0.3/DataFrames/src/dataframe/io.jl:731 > in readtable at /Users/rob/.julia/v0.3/DataFrames/src/dataframe/io.jl:812 > in readtable at /Users/rob/.julia/v0.3/DataFrames/src/dataframe/io.jl:879 > > julia> df = readtable("schools8_samples.csv", allowcomments=true) > 3x7 DataFrame > |-------|---------------|---------|---------| > | Col # | Name | Eltype | Missing | > | 1 | lp__ | Float64 | 0 | > | 2 | accept_stat__ | Float64 | 0 | > | 3 | stepsize__ | Float64 | 0 | > | 4 | treedepth__ | Int64 | 0 | > | 5 | n_leapfrog__ | Int64 | 0 | > | 6 | n_divergent__ | Int64 | 0 | > | 7 | mu | Float64 | 0 | > > > <schools8_samples.csv> >
test_df.jl
Description: Binary data
# diagnostic_file = kid_diagnostics.csv # refresh = 100 (Default) lp__,accept_stat__,stepsize__,treedepth__,n_leapfrog__,n_divergent__,beta.1,beta.2,beta.3,sigma -1470.54,0.997637,0.101195,5,23,0,27.7017,5.97474,0.549931,18.102 -1470.59,0.882989,0.101195,5,31,0,29.0516,5.48293,0.531079,18.2098 -1471.24,0.908469,0.101195,5,27,0,20.6426,3.84652,0.632324,18.3995 -1470.92,0.976811,0.101195,5,31,0,20.7159,7.44759,0.598549,18.1661
# diagnostic_file = kid_diagnostics.csv # refresh = 100 (Default) lp__,accept_stat__,stepsize__,treedepth__,n_leapfrog__,n_divergent__,beta.1,beta.2,beta.3,sigma # Adaptation terminated # Step size = 0.101195 # Diagonal elements of inverse mass matrix: # 37.6103, 5.11012, 0.00394028, 0.00110933 -1470.54,0.997637,0.101195,5,23,0,27.7017,5.97474,0.549931,18.102 -1470.59,0.882989,0.101195,5,31,0,29.0516,5.48293,0.531079,18.2098 -1471.24,0.908469,0.101195,5,27,0,20.6426,3.84652,0.632324,18.3995 -1470.92,0.976811,0.101195,5,31,0,20.7159,7.44759,0.598549,18.1661
# diagnostic_file = kid_diagnostics.csv # refresh = 100 (Default) lp__,accept_stat__,stepsize__,treedepth__,n_leapfrog__,n_divergent__,beta.1,beta.2,beta.3,sigma # Adaptation terminated # Step size = 0.101195 # Diagonal elements of inverse mass matrix: # 37.6103, 5.11012, 0.00394028, 0.00110933 -1470.54,0.997637,0.101195,5,23,0,27.7017,5.97474,0.549931,18.102 -1470.59,0.882989,0.101195,5,31,0,29.0516,5.48293,0.531079,18.2098 -1471.24,0.908469,0.101195,5,27,0,20.6426,3.84652,0.632324,18.3995 -1470.92,0.976811,0.101195,5,31,0,20.7159,7.44759,0.598549,18.1661
julia> include("/Users/rob/.julia/v0.3/MCMCExampleRepository/test/test_df.jl") 4x10 DataFrame |-------|---------------|---------|---------| | Col # | Name | Eltype | Missing | | 1 | lp__ | Float64 | 0 | | 2 | accept_stat__ | Float64 | 0 | | 3 | stepsize__ | Float64 | 0 | | 4 | treedepth__ | Int64 | 0 | | 5 | n_leapfrog__ | Int64 | 0 | | 6 | n_divergent__ | Int64 | 0 | | 7 | beta_1 | Float64 | 0 | | 8 | beta_2 | Float64 | 0 | | 9 | beta_3 | Float64 | 0 | | 10 | sigma | Float64 | 0 | 4x10 DataFrame |-------|---------------|---------|---------| | Col # | Name | Eltype | Missing | | 1 | lp__ | Float64 | 0 | | 2 | accept_stat__ | Float64 | 0 | | 3 | stepsize__ | Float64 | 0 | | 4 | treedepth__ | Int64 | 0 | | 5 | n_leapfrog__ | Int64 | 0 | | 6 | n_divergent__ | Int64 | 0 | | 7 | beta_1 | Float64 | 0 | | 8 | beta_2 | Float64 | 0 | | 9 | beta_3 | Float64 | 0 | | 10 | sigma | Float64 | 0 | ERROR: BoundsError() in findcorruption at /Users/rob/.julia/v0.3/DataFrames/src/dataframe/io.jl:663 in readtable! at /Users/rob/.julia/v0.3/DataFrames/src/dataframe/io.jl:731 in readtable at /Users/rob/.julia/v0.3/DataFrames/src/dataframe/io.jl:812 in readtable at /Users/rob/.julia/v0.3/DataFrames/src/dataframe/io.jl:879 in include at boot.jl:244 while loading /Users/rob/.julia/v0.3/MCMCExampleRepository/test/test_df.jl, in _expression_ starting on line 11 julia> On May 20, 2014, at 6:36 PM, Rob J. Goedman <[email protected]> wrote:
|
