Do rockets really need to hit a magic speed to escape Earth?

Not quite. Escape velocity, the minimum speed needed to break free of a planet's gravity, assumes a coasting, ballistic object with nothing else pushing it, like a cannonball. A rocket under continuous or intermittent thrust can escape at almost any speed, as long as it keeps pushing; what doesn't change is the minimum energy the trip requires. Newton's cannon thought experiment shows the coasting case: fire a cannonball fast enough and it circles the Earth, faster and its orbit stretches into an ellipse, and at exactly escape velocity the path becomes a parabola that only closes at infinity.

From Earth's surface that speed is 11.186 kilometres per second, about 40,270 km/h. The Moon, with about a sixth of Earth's surface gravity, needs only 2.38 km/s. Direction matters too, because Earth spins: a rocket launched eastward from the equator needs about 10.735 km/s relative to the ground, while one launched westward needs about 11.665 km/s. That's why launch sites are often placed as close to the equator as feasible.

Real rockets don't try to reach 11.2 km/s at ground level anyway: in a thick atmosphere, such hypersonic speeds would make most objects burn up from aerodynamic heating or tear apart from drag. Instead a spacecraft accelerates steadily out of the atmosphere until it reaches the escape velocity for its altitude, which is lower than at the surface.

Often it first parks in low Earth orbit, where it is already moving at about 7.8 km/s. From there, the extra change in speed needed to break free is far less than from a standing start, because most of the speed is already banked.