AWS S3 Security for folders under common bucket : Image courtesy freepik.com

Restricting access to different folders for different IAM accounts under a common Amazon S3 bucket

We at Inqude develop Membership and Event Management software called Gumpu ( https://mygumpu.com) . We use AWS S3 to store images and other files. Our product goes through multiple stages of testing and for each of these stages we use a common S3 bucket with different folders under it for each of the stages. Each stage content of the automation test is run on different machines with different IAM users as owners.

It was our intention to use a single S3 bucket since it makes it easy to define bucket names as variables, but use different folders as stage data something like this

<bucket>/stage1 ---> with access to IAM user 1
<bucket>/stage2 ---> with access to IAM user 2
<bucket>/stage3 ---> with access to IAM user 3

To achieve something like the one mentioned above we had to go through 2 step procedure

1. Create policies for each user and assign the policy giving it a right to access the bucket. Something like this for an IAM user to whom stage-1 folder is assigned under the common S3 bucket.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::bucket/stage-1",
                "arn:aws:s3:::bucket/stage-1/*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:DeleteObject"
            ],
            "Resource": [
                "arn:aws:s3:::bucket/stage-1",
                "arn:aws:s3:::bucket/stage-1/*"
            ]
        }
    ]
}

2. Create a bucket policy that restricts each user's access to the folder. The tricky part though is here. S3 bucket policy does not consider folders as resources. Thus making it difficult to restrict access to different folders under a bucket to a different user. Fortunately, AWS allows us to define conditions for the bucket policy that helps us restrict based on the folders

{
    "Version": "2012-10-17",
    "Id": "CCCCCPol987878",
    "Statement": [
        {
            "Sid": "Stmt1546414471931",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::ROOTID-1:user/stage1-user"
            },
            "Action": "s3:ListBucket",
            "Resource": "arn:aws:s3:::bucket",
            "Condition": {
                "StringEquals": {
                    "s3:prefix": [
                        "",
                        "stage1/"
                    ],
                    "s3:delimiter": "/"
                }
            }
        },
        {
            "Sid": "Stmt1546414471931",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::ROOTID-2:user/stage2-user"
            },
            "Action": "s3:ListBucket",
            "Resource": "arn:aws:s3:::bucket",
            "Condition": {
                "StringEquals": {
                    "s3:prefix": [
                        "",
                        "stage2/"
                    ],
                    "s3:delimiter": "/"
                }
            }
        }
    ]
}

Let me know if you come across a better solution to solving this problem.