Summary: If you're using Typelevel Scala, and http4s more specifically, http4s-aws can be a viable alternative to the AWS SDK for Java and does bring some benefits. Please give it a try and let us know if you spot any mistakes or opportunities for improvement.
If you're using Scala on AWS, you're likely using the official AWS SDK for Java. The SDK is probably the right choice for most applications, since it helps to quickly get started with AWS services. But if you're in the Typelevel ecosystem, you're probably using http4s, and might be wondering what an alternative looks like. In this post, we'll explore http4s-aws as an alternative to the AWS SDK.
Let's start with a motivating example. Arguably, one of the most common things to do is to fetch a key from Amazon S3. If we look at the GetObject REST API documentation, we might think that we should be able to write code like the following.
def getObject(client: Client[IO]): IO[String] =
for {
uri <- Uri.fromString(s"https://$bucket.s3.$region.amazonaws.com/$key").liftTo[IO]
contents <- client.expect[String](Request(Method.GET, uri))
} yield contents
If we're using http4s-aws, this is in fact exactly what we can do (if we
remember
key encoding). The thing missing in the example is the
Authorization header. And that is what http4s-aws provides.
In using the AwsSigningClient middleware, we get a
Client which deals with the signing logic and the rest mostly
works like you would expect, including streaming response bodies.
Using AwsSigningClient is fairly straightforward:
val client: Resource[IO, Client[IO]] =
for {
client <- EmberClientBuilder.default[IO].build
provider <- CredentialsProvider.default[IO]
} yield AwsSigningClient(provider, Region.EU_WEST_1, AwsServiceName.S3)(client)
while the credentials to use are managed by the
CredentialsProvider. What if we want to store objects in Amazon S3? Again, looking at the
PutObject
REST API documentation, we might want to write something along the
following lines.
def putObject(client: Client[IO]): IO[Unit] =
for {
uri <- Uri.fromString(s"https://$bucket.s3.$region.amazonaws.com/$key").liftTo[IO]
_ <- client.expect[Unit](Request(Method.PUT, uri).withEntity(content))
} yield ()
Again, this works like you would expect. If we want to stream the request body, http4s-aws also implements signed chunked uploads. Assuming you know the length of the request body, it is possible to do something like the following to upload the request body in chunks.
Request[IO](Method.PUT, uri)
.withHeaders(
contentType,
`Transfer-Encoding`(TransferCoding.chunked),
`X-Amz-Decoded-Content-Length`(contentLength)
)
.withBodyStream(content)
And if that is not enough, the http4s-aws-s3 module also implements multipart uploads. There's also support for pre-signing requests, which can be useful when deferring the upload to later on.
At this point, you might think "that looks nice and all, but using the AWS
SDK for Java is not so bad". It might be helpful to remind ourselves how a
GetObject request looks like with the AWS SDK for Java. Note
that it's possible to hide this kind of code in a library (pure-aws
is an example).
val client: Resource[IO, S3AsyncClient] =
Resource.fromAutoCloseable(IO(S3AsyncClient.builder().region(region).build()))
def getObject(client: S3AsyncClient): IO[GetObjectResponse] = {
val request = GetObjectRequest.builder().bucket(bucket).key(key).build()
IO.fromCompletableFuture(IO(client.getObject(request, AsyncResponseTransformer.toBytes[GetObjectResponse]())))
}
The example above doesn't deal with streaming or decoding of the response body, but gives you an idea of how a request works. Let's try to break down why you might not want to use the AWS SDK for Java, and what you get by instead using http4s-aws.
CredentialsProvider
is implemented in a non-blocking fashion on top of cats-effect and
http4s and the
default
supports most credential sources, like the AWS SDK. There is also an
integration
with the
Security Token Service (STS)
and AWS CLI for writing CLI
applications.
While http4s-aws is less battle-tested than the AWS SDK for Java, we're successfully running many production services using it. Most services do not need to access a lot of different AWS APIs, and using them is mostly trivial with http4s-aws.
There can initially be a bit more effort in reading the REST API documentation and implementing requests and response-handling (compared to using the AWS SDK), but ultimately it provides full flexibility, leaving you in control. Some of the more complex interactions, like multipart uploads, are supported by the library, but you are also free to implement your own variations.
Hopefully you find http4s-aws useful. Let us know about mistakes and ideas for improvement.