by flickering do you mean the highlights behave wildly? what are the coordinates on your direction light?
sounds like it might similar to a problem i had which was solved by moving any 0 coordinates of the directional light to a tiny non-zero value.. good luck richie
