AWS Lambda unable to GET a file from S3

Edward Q. Bridges

Hi I have a lambda (python3.6) below that is unable to read a file from S3, even though the lambda is in a role that has unfettered permissions for S3 (IAM policy below).

The Lambda simply attempts to retrieve a file from S3 and write it to a temporary location. However it blocks on calling s3.Bucket() and times out (even with a timeout in the minutes).

What's really weird is that it's timing out without any exception, and not rejecting the call to s3.Bucket() with some kind of permission error.

This is pretty basic, but I'm at a loss to get this sorted out.

import boto3
from botocore import exceptions

def lambda_handler(event, context):
  key = event['image']
  bucket = event['bucket']
  tempfile = '/tmp/%s-%s' % (bucket, key)
  print('(p) bucket: %s::%s' % (bucket, key))
  print('(p) tempfile: %s' % tempfile)
  s3 = boto3.resource('s3')
  print('(p) resource intiialized')
  try:
    b = s3.Bucket(bucket) 
    print('(p) bucket info: %s [%s]' % (b.name, b.creation_date))
    b.download_file(prefixed_key, tempfile)
    print('(p} file downloaded to %s' % tempfile)
  except exceptions.ParamValidationError as e:
    return {"statusCode": 400, "body": 'ParamValidationError: [%s]' % e}
  except exceptions.ClientError as e:
    message = '[%s]: [%s]' % (e.response['Error']['Code'], e.response['Error']['Message'])
    return {"statusCode": e.response['ResponseMetadata']['HTTPStatusCode'], "body": message}
  print('(p) image downloaded from s3 and stored at: %s' % tempfile)
  return None

IAM Policy that the role has is:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::my_bucket",
                "arn:aws:s3:::my_bucket/*"
            ]
        }
    ]
}

Example logs:

22:42:43
START RequestId: 61c60167-839d-11e7-97b1-a772bbde2609 Version: $LATEST
START RequestId: 61c60167-839d-11e7-97b1-a772bbde2609 Version: $LATEST
22:42:43
(p) bucket: my_bucket::my_key
22:42:43
(p) tempfile: /tmp/my_bucket/my_key
22:42:43
(p) resource intiialized
22:43:13
END RequestId: 61c60167-839d-11e7-97b1-a772bbde2609
END RequestId: 61c60167-839d-11e7-97b1-a772bbde2609
Edward Q. Bridges

The issue was narrowed down to a misconfiguration of the VPC. I simply configured it to run outside of a VPC as I don't need that for now, and it worked.

Collected from the Internet

Please contact [email protected] to delete if infringement.

edited at
0

Comments

0 comments
Login to comment

Related

From Javascript

How to get contents of a text file from AWS s3 using a lambda function?

From Dev

AWS Lambda using Python, get .json file data from S3 Bucket and put it into DynamoDB

From Dev

How to parse an AWS S3 file from a Lambda function

From Dev

aws lambda python append to file from S3 object

From Dev

Unable to get results from Scrapy on AWS Lambda

From Dev

Unable to trigger AWS Lambda by upload to AWS S3

From Dev

MD-5 Checksum error when uploading a file to S3 from AWS lambda

From Java

How to load a pickle file from S3 to use in AWS Lambda?

From Dev

Trigger AWS lambda function using input parameter from json file uploaded to S3

From Dev

How to move a File from One folder to Another Folder in the same AWS S3 bucket using Lambda?

From Dev

How to play an audio file from S3 Bucket on AWS Lambda function using Node.js?

From Dev

Trigger AWS Lambda with input parameters using S3 event trigger from multiple file paths

From Dev

How to upload a WAV file to S3 from the /tmp folder in AWS Lambda using Node.js

From Dev

AWS Lambda@edge. How to read HTML file from S3 and put content in response body

From Dev

How to access css/js file from S3 in AWS lambda function

From Dev

How to read csv file from s3 bucket in AWS Lambda?

From Dev

How can I decode a .gz file from S3 using an AWS Lambda function?

From Dev

Retrieve a file from browser using API gateway, AWS LAMBDA and S3

From Dev

AWS Lambda can't retrieve file from an Amazon S3 Bucket in same network

From Dev

To push the file from lambda to s3

From Dev

AWS Lambda - Unable to perform an action if object exists in S3

From Dev

How to read an .xls file from AWS S3 using spark in java? And unable to read sheetName

From Dev

Unable to Parse JSON file from S3 in AWS Quick Sight

From Dev

Get "fatal error: Unable to locate credentials" when I'm copying file from S3 to EC2 using aws cli

From Dev

AWS Rekognition throws exception Unable to get image metadata from S3

From Dev

How to setup aws-sdk to get file content from a public file in s3 bucket?

From

Out of memory when downloading 460mb file from S3 to AWS lambda using go aws sdk

From Java

AWS Lambda and S3 - uploaded pdf file is blank/corrupt

From Java

Reading a file on aws s3 with a Java lambda

Related Related

  1. 1

    How to get contents of a text file from AWS s3 using a lambda function?

  2. 2

    AWS Lambda using Python, get .json file data from S3 Bucket and put it into DynamoDB

  3. 3

    How to parse an AWS S3 file from a Lambda function

  4. 4

    aws lambda python append to file from S3 object

  5. 5

    Unable to get results from Scrapy on AWS Lambda

  6. 6

    Unable to trigger AWS Lambda by upload to AWS S3

  7. 7

    MD-5 Checksum error when uploading a file to S3 from AWS lambda

  8. 8

    How to load a pickle file from S3 to use in AWS Lambda?

  9. 9

    Trigger AWS lambda function using input parameter from json file uploaded to S3

  10. 10

    How to move a File from One folder to Another Folder in the same AWS S3 bucket using Lambda?

  11. 11

    How to play an audio file from S3 Bucket on AWS Lambda function using Node.js?

  12. 12

    Trigger AWS Lambda with input parameters using S3 event trigger from multiple file paths

  13. 13

    How to upload a WAV file to S3 from the /tmp folder in AWS Lambda using Node.js

  14. 14

    AWS Lambda@edge. How to read HTML file from S3 and put content in response body

  15. 15

    How to access css/js file from S3 in AWS lambda function

  16. 16

    How to read csv file from s3 bucket in AWS Lambda?

  17. 17

    How can I decode a .gz file from S3 using an AWS Lambda function?

  18. 18

    Retrieve a file from browser using API gateway, AWS LAMBDA and S3

  19. 19

    AWS Lambda can't retrieve file from an Amazon S3 Bucket in same network

  20. 20

    To push the file from lambda to s3

  21. 21

    AWS Lambda - Unable to perform an action if object exists in S3

  22. 22

    How to read an .xls file from AWS S3 using spark in java? And unable to read sheetName

  23. 23

    Unable to Parse JSON file from S3 in AWS Quick Sight

  24. 24

    Get "fatal error: Unable to locate credentials" when I'm copying file from S3 to EC2 using aws cli

  25. 25

    AWS Rekognition throws exception Unable to get image metadata from S3

  26. 26

    How to setup aws-sdk to get file content from a public file in s3 bucket?

  27. 27

    Out of memory when downloading 460mb file from S3 to AWS lambda using go aws sdk

  28. 28

    AWS Lambda and S3 - uploaded pdf file is blank/corrupt

  29. 29

    Reading a file on aws s3 with a Java lambda

HotTag

Archive