I have already tried to find the problem of this client-side and I had no issue so that is why I am labeling it a server problem.
Basically if I leave a minecart I am no longer able to interact with entities. I can do so once more once I relog.
EDIT: I misspelled minecart -.-
I've been getting into 3D game dev without the use of libraries other than opengl and it is hard but awesome. Took me so long to figure out how to tile textures and a ton of other crap from scratch but I did it :D
It is not visible but I made an fps style camera where you can move the mouse...