Filling in a gap left as exercise to the reader in part ten of Ray Tracing in One Weekend.
We start with Snell's law, which states that the product of the refractive index \(\eta\) of the first material and the sine of the angle \(\theta\) between the normal \(\bm{n}\) and the incoming ray equals the product of the refractive index \(\eta'\) of the second material and the sine of the angle \(\theta'\) between the refracted ray and the normal (check the sketch in the book). $$ \eta \sin \theta = \eta' \sin \theta' $$ The normal in our implementation always points against the incoming ray. We don't care about the position and length of the rays here, so we'll just deal with the unit directions \(\bm{r}\) and \(\bm{r'}\) of the incoming and refracted ray.
Since \(\bm{r_\bot}\) and \(\bm{r'_\bot}\) are parallel and have the same direction, the same relation holds for the vectors themselves. $$ \eta \cdot \bm{r_\bot} = \eta' \cdot \bm{r'_\bot} $$ We solve for the only unknown \(\bm{r'_\bot}.\) $$ \bm{r'_\bot} = \frac{\eta}{\eta'} \cdot \bm{r_\bot} $$ The refractive indices are given by our materials, so we only need to compute \(\bm{r_\bot}.\) It's easier to calculate the component that is parallel to the normal, so we use our decomposition to get \(\bm{r_\bot} = \bm{r} - \bm{r_\parallel}.\) We'll project \(-\bm{r}\) onto \(\bm{n}\) to get \(-\bm{r_\parallel}\) (the normal points against the incoming ray). Both \(\bm{r}\) and \(\bm{n}\) are unit vectors, so the projection is simply \(-\bm{r_\parallel} = \cos \theta \cdot \bm{n}.\) We derived the formula for the perpendicular component of the refracted ray given in the book. $$ \bm{r'_\bot} = \frac{\eta}{\eta'} \left( \bm{r} + \cos \theta \cdot \bm{n} \right) $$ Again, because both \(\bm{r}\) and \(\bm{n}\) are unit vectors and the normal points against the ray, we get the cosine of theta directly from the dot product of the negative ray direction and the normal. $$ \cos \theta = -\bm{r} \cdot n $$
The parallel component of the refracted ray is a scaled version of the negative normal. We only need to find the scalar factor, that is its length \(\lVert \bm{r'_\parallel} \rVert\), with the Pythagorean theorem. $$ \lVert \bm{r'_\parallel} \rVert^2 + \lVert \bm{r'_\bot} \rVert^2 = 1 $$ This gives us the parallel component of the refracted ray from the book. $$ \bm{r'_\parallel} = - \sqrt{1 - \lVert \bm{r'_\bot} \rVert^2} \cdot \bm{n} $$