I had to roll back a deploy this morning because I updated the version of python used in a library, but the Dockerfile used by the kubernetes cluster in our environment only installed an older version of python. I had to:
update my code
find out from my coworkers what other part of our pipelines were dependent on our code so I could verify my change was good
test the dependent part of the pipeline by running a deploy
see the deploy failed
see in the console logs that it was because of python version
get help from a colleague who figured out that the old version of python was coming from an ancient Dockerfile
create a revert pull request and get it approved
run a fresh deploy to verify
(There are tons of such things; I'm only thinking of that one because it happened today. Version incompatability is really common. There's also "certificate expired" or "processes depended on library version with breaking changes in an essential security patch" or "us-east-1 was unreachable, etc., etc. And the cardinal rule: "it's always DNS".)
![[community profile]](https://www.dreamwidth.org/img/silk/identity/community.png){kind=small}
![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png)
branchandroot
(no subject)
Date: 2025-05-28 03:09 am (UTC)I had to roll back a deploy this morning because I updated the version of python used in a library, but the Dockerfile used by the kubernetes cluster in our environment only installed an older version of python. I had to:
(There are tons of such things; I'm only thinking of that one because it happened today. Version incompatability is really common. There's also "certificate expired" or "processes depended on library version with breaking changes in an essential security patch" or "
us-east-1was unreachable, etc., etc. And the cardinal rule: "it's always DNS".)