RCA

An automatic patch update to our php-fpm service was deployed which caused the service to crash on restart.

It was found that a configuration which was working working before the update was now causing the service to crash which caused 502s to be returned by our routing system.

A patch was made in our agent and it was deployed on all our infrastructure.

Timeline

7:10PM EST Increase in 502 was detected and an investigation started

7:43PM EST It was found that php-fpm received an update which was causing it to crash

8:11PM EST A fix was found

8:57PM EST The fix started being deployed

9:18PM EST The fix has been deployed on all our infrastructure