If you push the pedal forwards when it is pointing upwards you are applying torque to the steering-axis. If you push the pedal down when it is pointing forwards you are pushing along a vector parallel to the steering-axis, so no steering effect. Try to put more effort into pushing down. Hard...