🎰
You're a team lead who's just back from a vacation.
In the first standup after you return, here's the conversation with your teammate (T):
Teammate(T): Hey, I am blocked since I don't have AWS access credentials for the last 2 days. I am unable to proceed with any work.
You: Well, have you raised a request to get the credentials?
T: Yes, I already sent an email two days ago.
You: Okay, I'll get you the credentials. What do you need the credentials for, btw?
T: I need to write a bash script to backup some data and upload parts of it to S3.
You: Okay, so how far along are you with the script?
T: I wasn't able to start, since I don't have the credentials.
You: Umm, why do you need credentials if you're writing the bash script. You can do this locally, no? And you can plug-in the S3 part later.
T: I don't know. I need to log in to an EC2 machine to write the script. The backup file is only present on that machine.
You: Umm, can you download the backup file locally, write and test your script and then scp the script to the EC2?
T: No, the backup file is too big.
You: How big?
T: 200+GB I guess
You: And, afaik, you're trying to extract the file, pick some folders from this file, massage some data and upload it to S3, right?
T: Yeah, kind of.
You: Ok, does the backup file have some fixed folder structure in it?
T: Yes
You: Do you know the exact folder structure of the backup tar file? An output of `tree` command for example?
T: umm, I can find that out I guess.
You: And do you know which folders and files you want to extract and upload to S3?
T: Yes, But, where are you going with this?
You: So, you can
simulate a similar folder structure locally,
create a dummy tar file,
pass the tar as input to your script, and
develop and test your script LOCALLY.
T: Umm, I guess I can. But why can't I just get credentials for AWS?
You: 😒😒😒
_some time passes_ ⌛
You: Let me teach you how things work in real world.
You: I'll get you the creds for sure, but first, you need to learn to break your work into dependencies that you're absolutely blocked on and things you can (and should) still do, without being blocked.
You: In this case, you can still proceed with writing the script with dummy data, locally, without any dependency.
You can also add lines to upload data to S3 (comment these out for now).
Accept the credentials from an env variables in your script. Handle error scenarios, etc..
You: Only when the time comes to deploy your script on prod, you need the credentials. Not before.
Learn to develop scripts and programs locally, in isolated manner.
Some mandatory reading:
The Goal book (understand system constraints)
Queuing theory (how to schedule work)
I write such stories on software engineering. There's no specific frequency as I don't make up these.
