I figured out the cause of my problems with the debugger and the Jacobian evaluation - I had a silly memory corruption issue which is now fixed.
I am still having trouble with convergence. I am using a test problem of intermediate size, which has nearly identical Jacobians when comparing finite difference to hand-coded. If I run with -snes_type newtonls -snes_monitor -ksp_monitor -ksp_ksp_monitor -ksp_monitor_true_residual -ksp_converged_reason -snes_converged_reason, the very first SNES iteration fails to converge, due to the KSP failing to converge after 10,000 iterations. Following the "Why is my iterative solver not converging?" FAQ, I ran the problem on an even smaller grid, with -snes_type newtonls -snes_monitor -ksp_monitor -ksp_ksp_monitor -ksp_monitor_true_residual -ksp_converged_reason -snes_converged_reason -pc_type svd -pc_svd_monitor, and the linear system was not close to singular. I don't have any reason to suspect that the linear system becomes more singular as the grid size increases. Next, I ran with -snes_type newtonls -snes_monitor -ksp_monitor -ksp_ksp_monitor -ksp_monitor_true_residual -ksp_converged_reason -snes_converged_reason -ksp_gmres_restart 1000 -pc_type none, and the KSP seemed to converge better, but the SNES only ran for a few iterations before it stopped with "Nonlinear solve did not converge due to DIVERGED_LINE_SEARCH iterations 7". Can I conclude from this that running the KSP without a pre-conditioner is a good idea? At that point, should I go back to asking why the SNES doesn't converge, ala the "Why is Newton's method not converging?" FAQ? Thanks for the continuing help!
