Endpoints

Duplication reduction

POST   /items       myshop.Items.create
GET    /items/:id   myshop.Items.read(id)

We want to abstract out the “/items” path prefix
We want to abstract out the myshop.Items value

Name binding

POST   /items       myshop.Items.create
GET    /items/:id   myshop.Items.read(id)

We want to abstract out the “/items” path prefix
We want to abstract out the myshop.Items value

val itemsPath = /items
val itemsCtl = myshop.Items

POST   itemsPath      itemsCtl.create
GET    itemsPath/:id  itemsCtl.read(id)

Name binding

POST   /items       myshop.Items.create
GET    /items/:id   myshop.Items.read(id)

We want to abstract out the “/items” path prefix
We want to abstract out the myshop.Items value

val itemsPath = path / "items"
val itemsCtl = myshop.Items

val create = endpoint(post(itemsPath),                   itemsCtl.create)
val read   = endpoint(get (itemsPath / segment[String]), itemsCtl.read _)

Abstract over a controller instance

What if the Items controller is a class, instead of an object?
We can not therefore statically reference this value
We need to abstract over this value

class ItemsEndpoints(itemsCtl: Items) {
  val itemsPath = path / "items"

  val create = endpoint(post(itemsPath),                   itemsCtl.create)
  val read   = endpoint(get (itemsPath / segment[String]), itemsCtl.read _)
}

Our router becomes a class itself and takes the controller as a constructor parameter (for instance)

Seamless integration with existing code

What if our app captures the concept of a resource as something that has two endpoints, create and read?

trait ResourceEndpoints {
  def create: Endpoint
  def read: Endpoint
}

We want the router to implement this trait

Seamless integration with existing code

trait ResourceEndpoints {
  def create: Endpoint
  def read: Endpoint
}

class ItemsEndpoints(itemsCtl: Items) extends ResourceEndpoints {
  val itemsPath = path / "items"

  val create = endpoint(post(itemsPath),                   itemsCtl.create)
  val read   = endpoint(get (itemsPath / segment[String]), itemsCtl.read _)
}

First-class values: summary

Defining the router and its constituent parts using first-class values liberates us from several constraints
- First-class values can be reused, combined, abstracted out, abstracted over, etc.
The price to pay is a slightly more cluttered syntax, though…

Multiple interpretations

We want to reuse the router’s knowledge to derive (for instance) an HTTP client consistent with the routes ;
We need to decouple the router from the application.

Going from a router to an HTTP protocol specification

class ItemsEndpoints(itemsCtl: Items) {
  val itemsPath = path / "items"

  val create = endpoint(post(itemsPath),                   itemsCtl.create)
  val read   = endpoint(get (itemsPath / segment[String]), itemsCtl.read _)
}

Let’s remove everything that’s related to the server implementation…

What else do we want to include in our specification?

object ItemsEndpoints {
  val itemsPath = path / "items"

  val create = endpoint(post(itemsPath))
  val read   = endpoint(get (itemsPath / segment[String]))
}

Let’s complete the specification with the HTTP entities…

Complete HTTP protocol specification

object ItemsEndpoints {
  val itemsPath = path / "items"

  val create = endpoint(post(itemsPath, jsonRequest[CreateItem]), jsonResponse[Item])
  val read   = endpoint(get(itemsPath / segment[UUID]), jsonResponse[Item])
}

case class Item(id: UUID, name: String, price: Int)

case class CreateItem(name: String, price: Int)

Deriving useful programs from a specification

The methods endpoint, post, path, jsonRequest, etc. each return an abstract representation of a part of the specification
Then, we want to give them several interpretations

HTTP server

class ItemsRouter(items: Items) {

  val router = playRouterFromEndpoints(
    playRoute(ItemsEndpoints.create, items.create),
    playRoute(ItemsEndpoints.read, items.read)
  )

}

A server decodes and extracts data from incoming requests, when they match the defined endpoints, and builds responses

HTTP server

class ItemsRouter(items: Items) {

  val router = playRouterFromEndpoints(
    playRoute(ItemsEndpoints.create, items.create),
    playRoute(ItemsEndpoints.read, items.read)
  )

}

class Items {
  def create(createItem: CreateItem): Future[Item] = …
  def read(id: UUID): Future[Item] = …
}

Note that the server implementation (the Items class) directly manipulate DTOs instead of JSON

HTTP client

class ItemsClient(wsClient: WSClient) {
  val create = playClient(ItemsEndpoints.create, wsClient)
  val read = playClient(ItemsEndpoints.read, wsClient)
}

A client builds requests from data and decodes responses

HTTP client

class ItemsClient(wsClient: WSClient) {
  val create = playClient(ItemsEndpoints.create, wsClient)
  val read = playClient(ItemsEndpoints.read, wsClient)
}

val client: ItemsClient = …
for {
  itemCreated <- client.create(CreateItem("foo", 42))
  itemRead <- client.read(itemCreated.id)
} yield itemRead

The client is non blocking and manages the JSON marshalling process

(Another) HTTP client

object ItemsClient {
  val create = xhrClient(ItemsEndpoints.create)
  val read = xhrClient(ItemsEndpoints.read)
}

val eventuallyItemCreated: js.Promise[Item] =
  ItemsClient.create(CreateItem("foo", 42))

A JavaScript client, statically typed and consistent with the server implementation (as long as they share the same specification).

Extensibility

We’ve already seen that it is desirable to have several interpretations of endpoints definitions
But what if we also need to extend the vocabulary used to define these endpoints?

Adding support for authentication

val create = endpoint(
  post(path / "items", jsonRequest[CreateItem]),
  jsonResponse[Item]
)
val read = endpoint(
  get(path / "items" / segment[UUID]),
  jsonResponse[Item]
)

We want to indicate, here, that the GET and POST requests must provide the appropriate credentials (in a way that is specific to our app – e.g. using HTTP basic authentication, or an API token, or a cookie, etc.)

Adding support for authentication

val create = endpoint(
  authenticatedPost(path / "items", jsonRequest[CreateItem]),
  jsonResponse[Item]
)
val read = endpoint(
  authenticatedGet(path / "items" / segment[UUID]),
  jsonResponse[Item]
)

So, we want to introduce the authenticatedGet and authenticatedPost terms (methods) to the mix
- And we want to define how these terms will be interpreted and to integrate them with existing interpretations

Abstract descriptions and multiple interpreters

`EndpointsAlg` interface

trait EndpointsAlg {
  def get[A](url: Url[A]): Request[A]
  type Url[A]
  type Request[A]
  …
}

EndpointsAlg provides vocabulary for defining HTTP protocol specifications:
- Terminals are defined by abstract methods,
- Information is carried by abstract type members,
Interpreters implement these methods and types with their own semantics ;
EndpointAlg has no JVM-only dependency, so it can also be used with Scala.js.

What is the “carried information”?

Information of interest that can vary for a given endpoint specification
E.g. a request that takes a user name as a query string parameter can be modeled as a Request[String]
- From a client perspective, String is the information to supply in order to build such a request ;
- from a server perspective, String is the information that is provided when processing such a request.
E.g. a request that takes a user name and age as query string parameters can be modeled as a Request[(String, Int)]

Client interpreter

trait PlayClient extends EndpointsAlg {

}

Client interpreter

trait PlayClient extends EndpointsAlg {

  type Request[A] = A => Future[WSResponse]

}

A Request[A] is a function that, given an A, eventually returns an HTTP response

Client interpreter

trait PlayClient extends EndpointsAlg {

  type Request[A] = A => Future[WSResponse]

  type Url[A] = A => String

}

An Url[A] is a function that, given an A, returns an URL as a String

Client interpreter

trait PlayClient extends EndpointsAlg {

  type Request[A] = A => Future[WSResponse]

  type Url[A] = A => String
  
  def get(url: Url[A]): Request[A] =
    a => wsClient.url(url(a)).get()

  def wsClient: WSClient
    
}

get takes the URL function as parameter, and returns a function that, given an A computes the URL and performs an HTTP request using an HTTP client

Server interpreter

trait PlayRouter extends EndpointsAlg {

}

Server interpreter

trait PlayRouter extends EndpointsAlg {

  type Request[A] = RequestHeader => Option[BodyParser[A]]

}

A Request[A] is a function that, given an incoming request’s headers, optionally returns a BodyParser[A] (defining how to get an A out of the request) or None if the incoming request does not match the request specification

Server interpreter

trait PlayRouter extends EndpointsAlg {

  type Request[A] = RequestHeader => Option[BodyParser[A]]

  type Url[A] = RequestHeader => Option[A]

}

An Url[A] is a function that, given an incoming request’s headers, optionally returns an A, or None if the incoming request does not match the request specification ;
Said otherwise, an Url[A] attempts to extract an A from request headers.

Server interpreter

trait PlayRouter extends EndpointsAlg {

  type Request[A] = RequestHeader => Option[BodyParser[A]]

  type Url[A] = RequestHeader => Option[A]

  def get(url: Url[A]): Request[A] =
    request =>
      if (request.method == "GET")
        url(request)
          .map(a => BodyParser(_ => Accumulator.done(Right(a))))
      else None

}

get takes the URL function as parameter and returns a function that, given an incoming request’s headers, checks that the request method is GET, applies the URL function, and finally returns a BodyParser that immediately succeeds with the A decoded from the URL

Interpreters: summary

An interpreter is a trait that extends EndpointsAlg and implements the type members and methods according to the desired semantics
Interpreters can be mixed in endpoints definitions to give them the desired semantics

Retroactive addition of new vocabulary

Authenticated requests

trait AuthRequests extends EndpointsAlg {
  def authenticatedGet[A](url: Url[A]): Request[(A, String)]
  …
}

New terms are defined as abstract methods, in a trait extending EndpointsAlg

Client interpreter of authenticated requests

trait AuthRequestsClient extends EndpointsAlg with PlayClient {
  def authenticatedGet[A](url: Url[A]): Request[(A, String)] = {
    case (a, apiToken) =>
      wsClient
        .url(url(a))
        .withHeaders("X-ApiToken" -> apiToken)
        .get()
  }
}

Server interpretation of authenticated requests

trait AuthRequestsRouter extends EndpointsAlg with PlayRouter {
  def authenticatedGet[A](url: Url[A]): Request[(A, String)] =
    request =>
      for {
        _ <- if (request.method == "GET") Some(()) else None
        a <- url(request)
        apiToken <- request.headers.get("X-ApiToken")
      } yield BodyParser(_ => Accumulator.done(Right((a, apiToken))))
}

Extensibility: summary

New terms are introduced as abstract methods, in a trait that extends EndpointsAlg
Interpreters implement them

1. Define your HTTP protocol specification

trait ItemsEndpoints extends EndpointsAlg with AuthRequests {
  val create = endpoint(
    authenticatedPost(path / "items", jsonRequest[CreateItem]),
    jsonResponse[Item]
  )
  val read = endpoint(
    authenticatedGet(path / "items" / segment[UUID]),
    jsonResponse[Item]
  )
}

case class Item(id: UUID, name: String, price: Int)

object Item {
  implicit val oformat: OFormat[Item] = …
}

case class CreateItem(name: String, price: Int)

object CreateItem {
  implicit val oformat: OFormat[CreateItem] = …
}

2. Implement the server

class ItemsRouter(items: ItemsService) extends ItemsEndpoints
  with PlayRouter with AuthRequestsRouter {

  val router = routerFromEndpoints(
    create.implementedBy(items.create),
    read.implementedBy(items.read)
  )

}

(Let’s pretend we have the following ItemsService class:)

class ItemsService {
  def create(apiToken: String, createItem: CreateItem): Item = …
  def read(apiToken: String, itemId: UUID): Item = …
}

3. Make the server executable

object Main extends App {
  val itemsService = new ItemsService
  val itemsRouter = new ItemsRouter(itemsService)
  NettyServer.fromRouter()(itemsRouter.router)
}

4. Define a client

val client =
  new ItemsEndpoints with PlayClient with AuthRequestsClient {
    val wsClient = …
  }

And then:

val eventuallyItem: Future[Item] =
  client.create("foo123", CreateItem("foo", 42))
…

Usage: summary

Define an abstract endpoints definition in a trait extending EndpointsAlg (or any other abstract trait that brings additional vocabulary) ;
Give it several semantics by mixing in the desired interpreters.

Comment tu exprimes un endpoint qui peut prendre plusieurs types de body, et qu’en fonction du body, tu peux avoir des types de retour différents ?

Antoine Michel, CTO, Zengularity.

Response type depending on the request type (rough idea)

trait ReqResp[A, B]

def myEndpoint[A, B](implicit
  reqResp: ReqResp[A, B],
  oformatA: OFormat[A],
  oformatB: OFormat[B]
): Endpoint[A, B] =
  endpoint(post(path / "foo", jsonRequest[A]), jsonResponse[B])

implicit val reqFooRespBar: ReqResp[CreateItem, ItemCreated] = null

implicit val reqBazRespQuux: ReqResp[UpdateItem, ItemUpdated] = null

How to control the HTTP response status?

Weak (but simple) solution:

trait PlayResults extends EndpointsAlg {
  type Result
  def playResult: Response[Result]
}

trait Foo extends PlayResults {
  val foo = endpoint(get(path / "foo"), playResult)
}

trait PlayResultsRouter extends PlayRouter {
  type Result = play.api.mvc.Result
  val playResult = identity
}

trait FooRouter extends Foo with PlayResultsRouter {
  val action = foo.implementedBy(_ => Ok("Hello"))
}

How to control the HTTP response status?

Better solution:

Define your own result type, according to your app requirements ;
For instance, for a JSON service:

case class Result[A](status: Int, entity: A)(implicit val owrites: OWrites[A])

Extend the EndpointsAlg trait:

trait MyAppResult extends EndpointsAlg {
  def result[A]: Response[Result[A]]
}

Implement a server interpreter:

trait MyAppResultRouter extends MyAppResult with PlayRouter {
  def result[A] = result => Status(result.status)(result.owrites(result.entity))
}

Object algebras (Oliveira, 2014)

Data types and folds

Consider the following data type definition:

sealed trait Expr
case class Add(lhs: Expr, rhs: Expr) extends Expr
case class Lit(value: Int) extends Expr

We can then represent expressions such as “1” or “1 + 2”: Lit(1), Add(Lit(1), Lit(2))

Data types and folds

Consider the following methods:

def eval(expr: Expr): Int =
  expr match {
    case Add(lhs, rhs) => eval(lhs) + eval(rhs)
    case Lit(x) => x
  }

def print(expr: Expr): String =
  expr match {
    case Add(lhs, rhs) => s"${print(lhs)} + ${print(rhs)}"
    case Lit(x) => x.toString
  }

They can be seen as interpreters for Expr ;
We observe that they share the same (recursive) structure. Let’s abstract it out!

Data types and folds

def fold[A](add: (A, A) => A, lit: Int => A)(expr: Expr): A =
  expr match {
    case Add(lhs, rhs) => add(fold(add, lit)(lhs), fold(add, lit)(rhs))
    case Lit(x) => lit(x)
  }

val eval: Expr => Int =
  fold[Int](_ + _, identity)

val print: Expr => String =
  fold[String]((lhs, rhs) => s"$lhs + $rhs", _.toString)

Data types and folds

Now, let’s apply some refactorings…

Fold-algebras

def fold[A](add: (A, A) => A, lit: Int => A)(expr: Expr): A =
  expr match {
    case Add(lhs, rhs) => add(fold(add, lit)(lhs), fold(add, lit)(rhs))
    case Lit(x) => lit(x)
  }

Let’s reify the parameters of fold into a proper type…

Fold-algebras

type FoldAlg[A] = ((A, A) => A, Int => A)

def fold[A](alg: FoldAlg[A]): Expr => A = {
  case Add(lhs, rhs) => alg._1(fold(alg)(lhs), fold(alg)(rhs))
  case Lit(x) => alg._2(x)
}

We say that FoldAlg is a fold algebra ;
Let’s use a trait with methods instead of a product of functions…

Object algebra interfaces

trait ExprAlg[A] {
  def add(lhs: A, rhs: A): A
  def lit(value: Int): A
}

def fold[A](alg: ExprAlg[A])(expr: Expr): A =
  expr match {
    case Add(lhs, rhs) => alg.add(fold(alg)(lhs), fold(alg)(rhs))
    case Lit(x) => alg.lit(x)
  }

We say that ExprAlg[A] is an object algebra interface

Object algebras

Let’s revisit the interpreters definition:

val eval: ExprAlg[Int] =
  new ExprAlg[Int] {
    def add(lhs: Int, rhs: Int) = lhs + rhs
    def lit(value: Int) = value
  }

val print: ExprAlg[String] =
  new ExprAlg[String] {
    def add(lhs: String, rhs: String) = s"$lhs + $rhs"
    def lit(value: Int) = value.toString
  }

We say that eval and print are object algebras

Getting rid of the intermediate data type using Church encodings

fold traverses an Expr and calls the corresponding methods of ExprAlg ;
Instead of using an intermediate Expr data type, let’s directly use ExprAlg as an abstract factory:

def onePlusTwo[A](alg: ExprAlg[A]): A =
  alg.add(alg.lit(1), alg.lit(2))

We also say that onePlusTwo is a Church encoding for Add(Lit(1), Lit(2)) ;
Then we can pass an interpreter to onePlusTwo, to get an actual value:

onePlusTwo(eval) == 3

onePlusTwo(print) == "1 + 2"

So what?

Inheritence can be used to achieve extensibility:

trait MulAlg[A] extends ExprAlg[A] {
  def mul(lhs: A, rhs: A): A
}

mul is defined independently of add and lit ;
That was not possible with algebraic data types!

Endpoints

Outline

Motivation

First-class values

Duplication reduction

Name binding

Name binding

Abstract over a controller instance

Seamless integration with existing code

Seamless integration with existing code

First-class values: summary

Multiple interpretations

Multiple interpretations

Going from a router to an HTTP protocol specification

What else do we want to include in our specification?

Complete HTTP protocol specification

Deriving useful programs from a specification

HTTP server

HTTP server

HTTP client

HTTP client

(Another) HTTP client

Extensibility

Extensibility

Adding support for authentication

Adding support for authentication

Implementation

Abstract descriptions and multiple interpreters

EndpointsAlg interface

What is the “carried information”?

Client interpreter

Client interpreter

Client interpreter

Client interpreter

Server interpreter

Server interpreter

Server interpreter

Server interpreter

Interpreters: summary

Retroactive addition of new vocabulary

Authenticated requests

Client interpreter of authenticated requests

Server interpretation of authenticated requests

Extensibility: summary

Usage

1. Define your HTTP protocol specification

2. Implement the server

3. Make the server executable

4. Define a client

Usage: summary

Examples

Response type depending on the request type (rough idea)

How to control the HTTP response status?

How to control the HTTP response status?

Questions?

Bonus

Object algebras (Oliveira, 2014)

Data types and folds

Data types and folds

Data types and folds

Data types and folds

Fold-algebras

Fold-algebras

Object algebra interfaces

Object algebras

Getting rid of the intermediate data type using Church encodings

So what?

Object algebras: summary

`EndpointsAlg` interface