AWS SQS — In a NutShell
2 min readApr 17, 2023
Overview
- SQS is a queue, e.g a Kinesis data streams producer can send messages to the SQS queue.
- Consumer will poll messages from the queue.
- Fully Managed.
- Very low latency, less than 10ms!
- No limit to how many messages can be in the queue.
- Default retention is 4 days, but this can be increased up-to 14 days.
- Can have duplicates messages (at least once delivery).
- Offers best effort ordering, but no guarantees.
- Messages have a limit of 256KB per message sent.
- Messages are defined in the body and are of type string.
- Can optionally provide a delay on delivery.
- Response back once a message has been sent to SQS queue. This will contain a message identifier and a MD5 hash of the body.
Consuming Messages
- Poll for messages, can do up-to 10 messages at a time.
- Consumer will delete the message once it has been consumed via the message ID & receipt handle. Therefore, messages cannot be processed by multiple applications.
FIFI Queue
- Names of the queue must end in .fifo
- Can send up-to 3,000 message/s compared to a standard queue with batching enabled and 300 without.
- Messages are processed in the exact order from the consumer that pushed them.
- Messages are sent exactly once.
- Can send large messages with SQS extended client (Java library). Producer will send the large payload 10GB to an s3 bucket and a small metadata message to the SQS queue, the consumer will then pull the metadata containing the bucket id to access the much larger payload.
Limitations
- Maximum of 120,000 in-flight messages being processed via the consumer.
- Max size per message 256KB.
- Retention from 1 minute to 14 days.
- Message content has to be of type String so either XML, JSON or text.
- Standard queues have no transactional per second (TPS), so no limit.
- FIFI queues default have a limit of 3,000 (batching).
- Pay per API request and network usage is well.
Security
- Encryption in flight using an HTTPS endpoint.
- Can enable server side encryption via KMS. Will only encrypt the body.
- IAM policy to allow usage of SQS.
- SQS queue access policy, offers more fine grained control such as over IP.