diff --git a/.env.example b/.env.example index b7276df..db73b71 100644 --- a/.env.example +++ b/.env.example @@ -1,5 +1,5 @@ -{ - "MFA_IDENTIFIER": "ARN", - "S3_ROLE": "ARN", - "SESSION_TYPE": "" -} \ No newline at end of file +MFA_IDENTIFIER="ARN", +S3_ROLE="ARN", +SESSION_TYPE="" +BW_AWS_ACCOUNT_SECRET_ID="" +BW_SESSION="" \ No newline at end of file diff --git a/lab-3/LAB-REPORT.md b/lab-3/LAB-REPORT.md index f317c75..c041021 100644 --- a/lab-3/LAB-REPORT.md +++ b/lab-3/LAB-REPORT.md @@ -7,10 +7,12 @@ - [x] Secrets/Token Management - [x] Consider secret-scanning - [x] Added git-leaks on pre-commit hook -- [x] Create & Connect to a Git*** repository - - [x] https://git.dropbear-minnow.ts.net/ +- [x] Create & Connect to a Git repository + - [x] https://code.wizards.cafe - [x] Modify and make a second commit - ![image of terminal](./assets/prep-console.png) + +![image of terminal](./assets/prep-console.png) + - [x] Test to see if gitea actions works - [x] Have an existing s3 bucket @@ -23,51 +25,82 @@ ## Lab - [x] create a custom IAM Policy - [x] create an IAM Role for EC2 - ![trust relationships](./assets/trust-relationships.jpg) - ![permissions](./assets/permissions.jpg) + +![trust relationships](./assets/trust-relationships.jpg) + +![permissions](./assets/permissions.jpg) + - [x] Attach the Role to your EC2 Instance - [x] Verify is3 access from the EC2 Instance * HTTPS outbound was not set up * I did not check outbound rules (even when the lab explicitly called this out) because it mentioned lab 2, so my assumption was that it had already been set up (it was not). When connection to s3 failed I double checked lab 3 instructions - ![screenshot of listing s3 contents](./assets/s3-access-screenshot.jpg) + +![screenshot of listing s3 contents](./assets/s3-access-screenshot.jpg) ### Stretch - [x] Create a bucket policy that blocks all public access but allows your IAM role - [ ] Implmented: [guide](https://aws.amazon.com/blogs/security/how-to-restrict-amazon-s3-bucket-access-to-a-specific-iam-role/) - ![restrict to role](./assets/restrict-to-role.jpg) -- [ ] **Experiment** with requiring MFA or VPC conditions. - - [ ] MFA conditions + +![restrict to role](./assets/restrict-to-role.jpg) + +- [x] **Experiment** with requiring MFA or VPC conditions. + - [x] MFA conditions * MFA did not work out of the box after setting it in the s3 bucket policy. The ways I found you can configure MFA: - * [stackoverflow](https://stackoverflow.com/questions/34795780/how-to-use-mfa-with-aws-cli) - * [official guide](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_use-resources.html) - * via cli roles - * configuration via ~/.aws/credentials - * 1Password CLI with AWS Plugin - * I use bitwarden, which also has an AWS Plugin - * This is probably what I will gravitate towards for a more - long-term setup, because having all of these credentials - floating around in various areas on my computer/virtualbox - envs gets confusing. Not a fan. - * I've seen a lot more recommendations (TBH it's more like 2 vs 0) - for 1password for password credential setup. Wonder why? - * other apps that handle this - - [ ] VPC + * [stackoverflow](https://stackoverflow.com/questions/34795780/how-to-use-mfa-with-aws-cli) + * [official guide](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_use-resources.html) + * [x] via cli roles - I set up a new set of role-trust relationships. + Update s3 Role: + Update action: sts:assumerole + Update principle (for user -- could not target group) + Add condition (MFA bool must be true) + * Commands referenced: I set up a script that looks like this -- [ ] **Host a static site** - - [ ] Enable a static website hosting (`index.html`) - - [ ] Configure route 53 alias or CNAME for `resume.` to the bucket endpoint. - - [ ] Deploy CloudFront with ACM certificate for HTTPS - #### Private "Innvite-Only" Resume Hosting - 1. **Pre-signed URLs** +```bash +MFA_TOKEN=$1 + +if [ -z "$1" ]; then + echo "Error: Run with MFA token!" + exit 1 +fi + +if [ -z $BW_AWS_ACCOUNT_SECRET_ID ]; then + echo "env var BW_AWS_ACCOUNT_SECRET_ID must be set!" + exit 1 +fi + +AWS_SECRETS=$(bw get item $BW_AWS_ACCOUNT_SECRET_ID) + +export AWS_ACCESS_KEY_ID=$(echo "$AWS_SECRETS" | jq -r '.fields[0].value') +export AWS_SECRET_ACCESS_KEY=$(echo "$AWS_SECRETS" | jq '.fields[1].value' | tr -d '"') + +SESSION_OUTPUT=$(aws sts assume-role --role-arn $S3_ROLE --role-session-name $SESSION_TYPE --serial-number $MFA_IDENTIFIER --token-code $MFA_TOKEN) +#echo $SESSION_OUTPUT +export AWS_SESSION_TOKEN=$(echo "$SESSION_OUTPUT" | jq '.Credentials.SessionToken' | tr -d '"') +export AWS_ACCESS_KEY_ID=$(echo "$SESSION_OUTPUT" | jq '.Credentials.AccessKeyId' | tr -d '"') +export AWS_SECRET_ACCESS_KEY=$(echo "$SESSION_OUTPUT" | jq '.Credentials.SecretAccessKey' | tr -d '"') +#echo $AWS_SESSION_TOKEN +#echo $AWS_ACCESS_KEY_ID +#echo $AWS_SECRET_ACCESS_KEY +aws s3 ls s3://witch-lab-3 +``` + + * configuration via ~/.aws/credentials + * 1Password CLI with AWS Plugin + * I use bitwarden, which also has an AWS Plugin + * I've seen a lot more recommendations (TBH it's more like 2 vs 0) + for 1password for password credential setup. Wonder why? + +- [x] **Host a static site** + - [x] Enable a static website hosting (`index.html`) + - [x] Configure route 53 alias or CNAME for `resume.` to the bucket endpoint. + - [x] Deploy CloudFront with ACM certificate for HTTPS + * see: [resume](https://resume.wizards.cafe) +- [ ] **Private "Invite-Only" Resume Hosting** + 1. [ ] **Pre-signed URLs** `aws s3 presign s3:///resume.pdf --expires-in 3600` - 2. **IAM-only access** - - [ ] Store under `private/` - - [ ] Write a bucket policy allowing only the role `EC2-S3-Access-Role-daphodell` to `GetObject` - 3. **Restrict to IP address** - - [ ] copy pasta json into bucket policy ### Further Exploration 1. [ ] Snapshots & AMIs @@ -80,7 +113,6 @@ - [ ] Bash: report world-writable files - [ ] Python with boto3: list snapshots, start/stop instances - ## Further Reading - [ ] - [ ] @@ -89,6 +121,14 @@ ## Reflection * What I built * Challenges + * Groups cannot be used as the principal in a trust relationship + * The stretch goal for setting up s3 + mfa was a bit of a pain: + * The earlier lab had me set up a trust relationship on the role to allow EC2 as a principal + on the role + When I later updated IAM permissions to include MFA, I promptly forgot about this detail + and had chatgpt help me with troubleshooting. It was pretty good at helping me figure out + the issue + * Security concerns On scale and security at scale @@ -107,5 +147,4 @@ classDef aside stroke-dasharray: 5 5, stroke-width:2px; - [ ] Clean up - [ ] Custom roles - [ ] Custom policies - - [ ] Stop ec2 Instance - - [ ] Remove s3 bucket \ No newline at end of file + - [ ] Shut down ec2 Instance \ No newline at end of file diff --git a/lab-3/LAB-REPORT.pdf b/lab-3/LAB-REPORT.pdf new file mode 100644 index 0000000..f0dfe02 Binary files /dev/null and b/lab-3/LAB-REPORT.pdf differ diff --git a/lab-3/assets/restrict-to-role.jpg b/lab-3/assets/restrict-to-role.jpg index 010512c..46ab6fe 100644 Binary files a/lab-3/assets/restrict-to-role.jpg and b/lab-3/assets/restrict-to-role.jpg differ diff --git a/lab-resume/LAB-REPORT.md b/lab-resume/LAB-REPORT.md new file mode 100644 index 0000000..accb434 --- /dev/null +++ b/lab-resume/LAB-REPORT.md @@ -0,0 +1,22 @@ +# [daphodell] +- [git](https://code.wizards.cafe) + +### Projects +- Placeholder + +## Skills +- Cloud Platforms & Security: AWS (IAM) +- Infrastructure as Code: Terraform, CloudFormation +- Programming and Scripting: JavaScript, Bash, Go +- DevOps & Automation: CI/CD Pipelines (GitHub Actions, Gitlab, Gitea Actions), Docker + +## Professional Experience +### Senior Software Engineer +[Company A], 2025 +- Automated data pipeline to ingest and transform data from unstructured municipal websites +- Leveraged LLMs to parse and normalize data +- Improved update frequency and data accuracy via integration and baseline testing. + +### Senior Software Engineer +[Company B], 2022-2024 +- Placeholder text diff --git a/lab-resume/LAB-REPORT.pdf b/lab-resume/LAB-REPORT.pdf new file mode 100644 index 0000000..8bf870e Binary files /dev/null and b/lab-resume/LAB-REPORT.pdf differ diff --git a/lab-resume/index.html b/lab-resume/index.html new file mode 100644 index 0000000..d32a115 --- /dev/null +++ b/lab-resume/index.html @@ -0,0 +1,157 @@ + + + + + daphodell Resume + + + + +
+

+ daphodell +

+ + + git + + + +
+

Skills

+
    +
  • Cloud Platforms & Security: AWS (IAM)
    • +
  • Infrastructure as Code: Terraform, CloudFormation
    • +
  • Programming and Scripting: JavaScript, Bash, Go
    • +
  • DevOps & Automation: CI/CD Pipelines (GitHub Actions, Gitlab, Gitea Actions), Docker
    • +
+
+ +
+

Professional Experience

+
+ Senior Software Engineer + [Company A] + 2025 +
    +
  • Automated data pipeline to ingest and transform data from unstructured municipal websites
    • +
  • Leveraged LLMs to parse and normalize data
    • +
  • Improved update frequency and data accuracy via integration and baseline testing.
    • +
+
+
+ Senior Software Engineer + [Company B] + 2022-2024 +
    +
  • Placeholder text
    • +
+
+
+
+ + diff --git a/mise.toml b/mise.toml index 1edf5d8..fa1c6b7 100644 --- a/mise.toml +++ b/mise.toml @@ -1,22 +1,20 @@ [tools] aws-cli = 'latest' -bitwarden = 'latest' -jq = 'latest' -bw = 'latest' +pandoc = 'latest' +# Also required: +# - bitwarden CLI +# - jq [env] -_.file = '.env.json' -BLAH = "{{ exec(command=\"bw get item $BW_AWS_ACCOUNT_SECRET_ID\") }}" +_.file = '.env' [tasks.ssh] run = "ssh -p 5679 vboxuser@127.0.0.1" -[tasks.generate] +[tasks.labgen] run = "./utilities/pdf_make/labs.sh" -[tasks.setup-aws] +[tasks.poke-s3] run = """ -export SECRETS_OBJECT=$(bw get item $BW_AWS_ACCOUNT_SECRET_ID) -export AWS_ACCESS_KEY_ID=$(echo "$SECRETS_OBJECT" | jq -r '.fields[0].value') -export AWS_SECRET_ACCESS_KEY=$(echo "$SECRETS_OBJECT" | jq '.fields[1].value') +./utilities/setup_aws/use-s3.sh """ \ No newline at end of file diff --git a/utilities/pdf_make/generate_reports.sh b/utilities/pdf_make/generate_reports.sh index 0376acd..cb1672e 100755 --- a/utilities/pdf_make/generate_reports.sh +++ b/utilities/pdf_make/generate_reports.sh @@ -21,7 +21,7 @@ find . -maxdepth 1 -type d -name "lab-*" | while read lab_dir; do # Generate PDF using pandoc # Make sure 'pandoc' command is available in the image, which it is for pandoc/latex image_dir="$lab_dir" - pandoc "$markdown_file" -s -o "$pdf_file" --pdf-engine=pdflatex --resource-path "$image_dir" + pandoc "$markdown_file" -s -o "$pdf_file" --pdf-engine=pdflatex --resource-path "$image_dir" -V geometry:margin=0.5in if [ $? -eq 0 ]; then echo "Successfully generated $pdf_file" diff --git a/utilities/setup_aws/use-s3.sh b/utilities/setup_aws/use-s3.sh new file mode 100755 index 0000000..7f481c5 --- /dev/null +++ b/utilities/setup_aws/use-s3.sh @@ -0,0 +1,29 @@ +#!/bin/bash + +MFA_TOKEN=$1 + +if [ -z "$1" ]; then + echo "Error: Run with MFA token!" + exit 1 +fi + +if [ -z $BW_AWS_ACCOUNT_SECRET_ID ]; then + echo "env var BW_AWS_ACCOUNT_SECRET_ID must be set!" + exit 1 +fi + +AWS_SECRETS=$(bw get item $BW_AWS_ACCOUNT_SECRET_ID) + +export AWS_ACCESS_KEY_ID=$(echo "$AWS_SECRETS" | jq -r '.fields[0].value') +export AWS_SECRET_ACCESS_KEY=$(echo "$AWS_SECRETS" | jq '.fields[1].value' | tr -d '"') + +SESSION_OUTPUT=$(aws sts assume-role --role-arn $S3_ROLE --role-session-name $SESSION_TYPE --serial-number $MFA_IDENTIFIER --token-code $MFA_TOKEN) +#echo $SESSION_OUTPUT +export AWS_SESSION_TOKEN=$(echo "$SESSION_OUTPUT" | jq '.Credentials.SessionToken' | tr -d '"') +export AWS_ACCESS_KEY_ID=$(echo "$SESSION_OUTPUT" | jq '.Credentials.AccessKeyId' | tr -d '"') +export AWS_SECRET_ACCESS_KEY=$(echo "$SESSION_OUTPUT" | jq '.Credentials.SecretAccessKey' | tr -d '"') +#echo $AWS_SESSION_TOKEN +#echo $AWS_ACCESS_KEY_ID +#echo $AWS_SECRET_ACCESS_KEY +aws s3 ls s3://witch-lab-3 +echo "finished!" \ No newline at end of file