Purpose of the valve is to release the pressure when done filling a zerk before moving on to the next one.
catdude, the lever on my Alemite pump does the same thing you're describing if I just pump grease out of the hose without it attached to anything. The pressure relief lever seems to work best when I'm hooked up to a fitting and there's a little presure on the system, then the valve vents the excess grease out of a little hole just inside the top of the pump, returning it for recirculation.