Hello Coherence, Part 1

Use the open source Oracle Coherence Community Edition to create stateful applications that are as easy to scale, if not easier, than the stateless applications you are building today.

Published in

Oracle Coherence

22 min readSep 5, 2020

This article was originally published in Java Magazine, on August 14, 2020. It has since been updated to cover the latest Coherence CE 22.06 and Helidon 2.5 releases.

Oracle Coherence is nearly 20 years old. It started as a distributed caching product and then evolved into an in-memory data grid. It’s an essential tool for improving the performance and scalability of Java EE applications, and it’s widely used for large-scale projects.

Think of it as a scalable, concurrent, fault-tolerant java.util.Map implementation that is partitioned across multiple JVMs, machines, and even data centers. That is a major oversimplification, of course, but it is sufficient for now.

For most of its history, Oracle Coherence was a commercial product with a fairly high price tag. That limited its appeal mainly to corporate users and took it out of consideration for many applications, including smaller or open source projects that didn’t absolutely need the features it offers.

All that changed on June 25, 2020, when Oracle released Coherence Community Edition (CE), an open source version of the product.

Coherence CE may often be the best option you have when building modern, cloud native applications and services. Along with my colleague Harvey Raja, I recently wrote A Gentle Introduction to Coherence, covering what the platform is and why you should consider using it for your next application, so I will not repeat that information here.

In this article, I will focus on how to build scalable stateful applications using Coherence CE.

Extending the example To Do List service application

In this article, I will extend the To Do List application used as a quick start example on the Coherence CE website (see Figure 1). I’m doing this for two reasons:

It has a very simple domain model that everyone understands, so this article can focus on the usage of Coherence.
Despite its simplicity, the sample application demonstrates many Coherence features from basic reads and writes, to queries and aggregations, to in-place processing and events.

Figure 1. Screenshot of the To Do List application

I’ll make my application a lot more interesting than the quick start example, though: In addition to the Helidon REST service implemented there, my project will do the following:

Add support for server-sent events (SSEs) to the REST API to broadcast Coherence events to REST clients.
Implement a React-based web front end that will use the Helidon REST API, which will be served by the Helidon Web Server as well.
Configure Coherence CE’s gRPC server and implement a JavaFX front end that uses the native Java client to interact with Coherence over gRPC.
Demonstrate how all the components above work together.
Deploy the application into a Kubernetes cluster using Coherence Operator.

Although I will use Helidon, JavaFX, and React to implement the application, the focus will be very much on Coherence CE usage and APIs. Everything else is secondary, but you are more than welcome to explore the source code for the complete application, which is available on GitHub.

Fair warning: I am not a JavaFX or React expert, so some of the UI-related code may be suboptimal.

As you can see from the list above, there’s too much to fit into a single article. So, I will implement the REST API back end for the application here, and I’ll leave front-end implementations and operational aspects, such as deployment and monitoring, for future articles in this series.

Implementing the REST API

The first step is to create a Maven project with all the needed Helidon and Coherence CE dependencies. Do that by following the instructions in the Helidon MP Tutorial, and then add a few additional dependencies.

The resulting POM file should look like this:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xmlns="http://maven.apache.org/POM/4.0.0"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>   <parent>
    <groupId>io.helidon.applications</groupId>
    <artifactId>helidon-mp</artifactId>
    <version>2.5.1</version>
    <relativePath/>
  </parent>  <groupId>com.oracle.coherence.examples</groupId>
  <artifactId>todo-list-helidon-server</artifactId>
  <version>1.0.0-SNAPSHOT</version>  <properties>
    <coherence.groupId>com.oracle.coherence.ce</coherence.groupId>
    <coherence.version>22.06.1</coherence.version>
  </properties>  <dependencies>
    <!-- Helidon dependencies-->
    <dependency>
      <groupId>io.helidon.microprofile.bundles</groupId>
      <artifactId>helidon-microprofile</artifactId>
    </dependency>
    <dependency>
      <groupId>org.glassfish.jersey.media</groupId>
      <artifactId>jersey-media-json-binding</artifactId>
    </dependency>
    <dependency>
      <groupId>org.glassfish.jersey.media</groupId>
      <artifactId>jersey-media-sse</artifactId>
    </dependency>    <!-- Coherence CE dependencies -->
    <dependency>
      <groupId>${coherence.groupId}</groupId>
      <artifactId>coherence-cdi-server</artifactId>
      <version>${coherence.version}</version>
    </dependency>
    <dependency>
      <groupId>${coherence.groupId}</groupId>
      <artifactId>coherence-mp-config</artifactId>
      <version>${coherence.version}</version>
    </dependency>
  </dependencies>  <build>
    <plugins>
      <plugin>
        <groupId>org.jboss.jandex</groupId>
        <artifactId>jandex-maven-plugin</artifactId>
        <executions>
          <execution>
            <id>make-index</id>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build></project>

In addition to the Helidon MicroProfile Bundle, I have added Eclipse Jersey support for JSON-B serialization and SSEs. The code also configures the Jandex plugin to index the application’s classes at build time to speed up Contexts and Dependency Injection (CDI) startup.

On the Coherence CE side, there are two added modules:

Coherence CDI Server, which provides a CDI extension that starts the Coherence CE server within the application process, enables the injection of Coherence maps into Helidon services, and maps Coherence events to CDI events so they can be handled using standard CDI observers
Coherence MicroProfile Config, which will configure Coherence CE using the Helidon MP Config implementation

I used a Maven property to specify not only the Coherence CE version but also the Maven groupId for Coherence CE dependencies. This is a recommended practice, because it allows you to easily switch between the open source version (Coherence CE) and the commercial versions (Oracle Coherence Enterprise Edition or Oracle Coherence Grid Edition), which have a different groupId. Everything else in your code, from artifact names to package and class names, can stay exactly the same.

In addition to creating a Maven project, it’s necessary to create a few configuration files. First, turn the application into a proper CDI bean archive by creating the META-INF/beans.xml file:

<?xml version="1.0" encoding="UTF-8"?><beans xmlns="http://xmlns.jcp.org/xml/ns/javaee"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/javaee
                         http://xmlns.jcp.org/xml/ns/javaee/beans_2_0.xsd"
        version="2.0"
        bean-discovery-mode="annotated">

It’s necessary to configure Java Logging to see log output from Helidon and Coherence CE. To configure it, add the logging.properties file to the src/main/resources directory:

handlers=io.helidon.common.HelidonConsoleHandler# Global default logging level. Can be overriden by specific handlers and loggers
.level=CONFIG# Formatter configuration
java.util.logging.SimpleFormatter.format=%1$tY.%1$tm.%1$td %1$tH:%1$tM:%1$tS %4$s %3$s !thread!: %5$s%6$s%n

Implementing the domain model

To implement the REST API for the To Do List application, the application will need two things: a data model representing a single task in a to-do list and a JAX-RS resource that provides the necessary REST endpoints.

The first one is a trivial plain old Java object (POJO) with a handful of attributes that are very much self-explanatory:

public class Task
        implements Serializable
    {
    private String id;
    private long createdAt;
    private String description;
    private Boolean completed;    /**
     * Construct Task instance.
     *
     * @param description  task description
     */
    public Task(String description)
        {
        this.id = UUID.randomUUID().toString().substring(0, 6);
        this.createdAt = System.currentTimeMillis();
        this.description = description;
        this.completed = false;
        }    // accessors omitted for brevity
    }

Note that this POJO is serializable using Java serialization, which is what Coherence CE will use as a storage format, and using JSON-B, which will be used by the REST and gRPC APIs. Coherence CE supports pluggable serializers for both storage and the client/server communication. I could have used JSON for both. The application could also have used JSON as a transport format and Coherence Portable Object Format (POF) as a storage format.

The goal here is to demonstrate that you can use different serialization formats for client/server communication and storage, with Coherence CE automatically converting objects as necessary. It is a bit too early to talk about Coherence POF, though; that will come up later.

Implementing Task Repository

In addition to a low-level NamedMap API, Coherence CE provides a higher level Repository API. To make data access code as simple as possible, we will implement TaskRepository:

@ApplicationScoped
public class TaskRepository extends AbstractRepository<String, Task>
    {
    @Inject
    private NamedMap<String, Task> tasks;    protected NamedMap<String, Task> getMap()
        {
        return tasks;
        }    protected String getId(Task task)
        {
        return task.getId();
        }    protected Class<? extends Task> getEntityType()
        {
        return Task.class;
        }
    }

The implementation above is as simple as it gets: we implement AbstractRepository, define the ID and entity types for it, and inject a NamedMap that will be used as a backing store for our repository implementation.

Coherence CE’s NamedMap is an extension of java.util.Map, an interface most developers know well. This is both good and bad. It is good because developers already know how to do many things with it. It is bad because developers may do things in a suboptimal way when a better alternative is readily available.

To dig deeper, unlike Java’s Map, Coherence CE’s NamedMap is a distributed data structure. Data in it is not stored within local buckets, as it is in a Java HashMap, for example. Instead, it is stored within partitions, which can be distributed across many JVMs, machines, or even data centers. This means that some of the operations you take for granted when you work with a local map are not nearly as efficient when you work with a NamedMap.

Take iteration as an example: You wouldn’t think twice about iterating over the entries of a HashMap. However, iterating over a NamedMap can be an extremely expensive operation, potentially moving gigabytes or even terabytes of data across the network, deserializing it, processing it, garbage collecting it, and so on.

Fortunately, most things work as you would expect, and the Coherence CE team has done a lot of work to make sure that’s the case. For example, the team has reimplemented many of the Map methods to use Coherence CE primitives, such as aggregators and entry processors, in a way that is suitable for a distributed Mapimplementation. Similarly, the developers have completely reimplemented the Stream API on top of Coherence CE aggregators, effectively allowing you to perform stream operations in parallel across many machines. Distributed lambdas have been reimplemented to allow them to be serialized using any of the supported serialization formats in a way that can be safely used in a distributed environment, with potentially different versions of those lambdas being present on different cluster members.

The point is that even though the Coherence CE developers have done all that optimization, you still need to remember that you are working with a distributed data structure and that most calls will end up on the network. Therefore, you should use Coherence CE features that allow you to minimize the impact of that, and Repository API is one of those features, as you’ll see in a minute.

Implementing the Service

With the data model and the repository implementations in place, we can start implementing business logic for our task management service.

While we could do this directly within our JAX-RS resource that will be used to provide REST endpoints for our service, that would not be ideal. We also want to provide gRPC and GraphQL endpoints, so implementing business logic in a way that allows us to reuse it for all three API implementations seems more appropriate.

@ApplicationScoped
public class ToDoListService
    {
    @Inject
    protected TaskRepository tasks;    // TBD
    }

First, the code needs some basic CRUD functionality: the ability to add, update, and remove individual tasks and to retrieve a list of all tasks.

public Task createTask(String description)
    {
    Objects.requireNonNull(description, "description is required");
    return tasks.save(new Task(description));
    }

The method above creates a new Task instance with a given description and saves it to the repository. The return value from the save method is fully populated instance of the created Task object.

The methods to find and remove existing tasks are almost as simple:

public Task findTask(String id)
    {
    return Optional
            .ofNullable(tasks.get(id))
            .orElseThrow(() -> new TaskNotFoundException());
    }public Task deleteTask(String id)
    {
    return Optional
            .ofNullable(tasks.removeById(id, true))
            .orElseThrow(() -> new TaskNotFoundException());
    }

The only complication is that we need to throw a TaskNotFoundException if the task with the specified ID doesn’t exist in the repository. This is an application exception, which will be mapped to an appropriate HTTP response (404 Not Found, in this case) using JAX-RS exception mapper.

Now let’s implement the logic that will allow us to fetch a list of tasks and to remove tasks based on their completion status:

public Collection<Task> findTasks(Boolean completed)
    {
    Filter<Task> filter = completed == null
                          ? always()
                          : equal(Task::getCompleted, completed);    return tasks.getAllOrderedBy(filter, Task::getCreatedAt);
    }public boolean deleteCompletedTasks()
    {
    return tasks.removeAll(isTrue(Task::getCompleted));
    }

This code shows an example of an optimization I mentioned earlier. Instead of fetching all the tasks from the server and filtering them on the client, I use TaskRepository.getAllOrderedBy method, which uses Coherence Filter API to execute the query in parallel across the cluster members and returns only the subset of the tasks with the specified completion status. It also sorts the results based on task creation time, so the results return to the clients in a consistent order.

If the client doesn’t specify the completion status to query on, we return all tasks by passing an AlwaysFilter instance to the method.

Similarly, we use TaskRepository.removeAll method with a Filter that evaluates to true if the task is completed, in order to remove all completed tasks from the repository in parallel.

Finally, let’s look at a slightly more interesting piece of functionality: mutating operations:

public Task updateDescription(String id, String description)
    {
    Task task = tasks.update(id, Task::setDescription, description);
    return Optional
            .ofNullable(task)
            .orElseThrow(() -> new NotFoundException());
    }public Task updateCompletionStatus(String id, boolean completed)
    {
    Task task = tasks.update(id, Task::setCompleted, completed);
    return Optional
            .ofNullable(task)
            .orElseThrow(() -> new NotFoundException());
    }

Just like with findTask and deleteTask, we throw a NotFoundException if the task with the specified ID doesn’t exist in the repository. However, that is not the “interesting” part — what is interesting is how the update itself is performed.

Unlike many data stores, Coherence doesn’t force you to use read-modify-write idiom in order to mutate data in the cluster — as a matter of fact, it discourages you from doing so.

For example, I could’ve written the code above as:

public Task updateDescription(String id, String description)
    {
    Task task = tasks.get(id);             // read
    if (task == null) 
        { 
        throw new NotFoundException(); 
        }    task.setDescription(description);      // modify
    return tasks.save(task);               // write
    }

But I didn’t, and you shouldn’t…

There are two problems withe the code above:

It is not correct from a concurrency perspective — multiple processes could retrieve the same task from the cluster, modify it in a different way, and write it back. The last one to write would win, and all the changes made by other processes would be lost. Sure, we could use explicit locking to avoid that, but that is a very expensive solution that turns a 6-network hop operation into a 14-network hop operation, as locking and unlocking require additional network calls to both primary and backup members.
It is inefficient even without locking, as it moves potentially large object over the wire three times, and requires 6 network hops on its own (2 for get and 4 for save).

What we did instead was this:

Task task = tasks.update(id, Task::setDescription, description);

There is a lot happening there, so let’s break it down:

Find the primary copy of the task with the specified id, wherever it may be in the cluster (likely on a remote cluster member)
Modify it by calling setDescription method on it, passing specified description as an argument
Return updated Task to the caller.

This way we don’t need explicit lock, as Coherence will obtain a lightweight, implicit lock on the primary copy of the object automatically before modifying it. It also reduces the number of times we move the object over the wire from three to two (from primary to backup, and from primary to the caller).

The update method used above barely scratches the surface of what you can do. You could also use one of more powerful update overloads to update multiple attributes in one call, create entity instance if it doesn’t exist, or even avoid sending the updated value back to the caller if you don’t need it, further reducing the amount of data moving over the wire.

Understanding Entry Processors
This feature of the Repository API builds on one of the most important distributed computing primitives you will ever use with Coherence: entry processors.
Entry processors allow you to send functions into the cluster and process data in place, which completely eliminates the need to move the data over the network in most cases. In other words, the function moves to the data, not the other way around!
To some extent, you can compare entry processors to stored procedures: entry processors efficiently process large data sets in parallel without moving any data over the network (apart from any processing results, which you can optionally return). However, there are two important differences between them:
1. Entry processors are written in Java instead of SQL, because Java is to Coherence what SQL is to a database, and this makes them much easier to write and use (for Java developers, at least) than stored procedures could ever be.
2. Entry processors can be predefined via an existing Java class implementation that is available on both the client and the server. Or, they can be created dynamically via Java lambdas, in which case they are shipped from the client to the server, bytecode and all, and registered as versioned classes to support multiple versions of the same lambda across the cluster. The example above uses a simple method reference, but I’ll show a lambda example shortly.
By the way, entry processors are guaranteed to execute exactly once, even in the case of failures of primary or backup members, as long as the primary member and all its backups do not fail at the same time. This is an incredibly difficult guarantee to provide in a distributed system, and it’s a huge differentiator between Coherence and products from some of Oracle’s competitors, who may have a similar feature without the same guarantees Coherence provides.
I cannot stress enough how important and powerful entry processors are. In theory, you could implement every other method in the NamedMap API using entry processors, and as a matter of fact, I did use them to implement many of the default Map methods that were added in Java 8, and most of the AsyncNamedMap API.

With that, we are done with the service implementation. It is time to put some API facades in front of it, so it can be used by REST, gRPC and GraphQL clients.

Implementing REST endpoints

Now that we have a service that provides the functionality we want to expose to the clients, writing a JAX-RS facade that will do that is trivial. But first, let’s define the API that we want to provide to the clients:

POST /api/tasks              # create a new task
GET  /api/tasks[?completed]  # fetch tasks based on completion state
DELETE /api/tasks/:id        # delete specified task
DELETE /api/tasks            # delete completed tasks
PUT  /api/tasks/:id          # update specified task
GET  /api/tasks/events       # register for events (SSE)

The first four of these endpoints do nothing more than delegate to our service:

@Path("/api/tasks")
@ApplicationScoped
public class ToDoListRestApi
    {
    @Inject
    private ToDoListService api;    @POST
    @Consumes(APPLICATION_JSON)
    public Task createTask(JsonObject task)
        {
        return api.createTask(task.getString("description"));
        }    @GET
    @Produces(APPLICATION_JSON)
    public Collection<? extends Task>  
    getTasks(@QueryParam("completed") Boolean completed)
        {
        return api.getTasks(completed);
        }    @DELETE
    @Path("{id}")
    public Task deleteTask(@PathParam("id") String id)
        {
        return api.deleteTask(id);
        }    @DELETE
    public boolean deleteCompletedTasks()
        {
        return api.deleteCompletedTasks();
        }    // TBD
    }

The functionality to update a task is slightly more complex, as we need to determine whether to update description or a completion status based on the PUT payload:

@PUT
@Path("{id}")
@Consumes(APPLICATION_JSON)
public Task updateTask(@PathParam("id") String id, JsonObject task)
    {
    if (task.containsKey("description"))
        {
        return api.updateDescription(
                      id, task.getString("description"));
        }
    else if (task.containsKey("completed"))
        {
        return api.updateCompletionStatus(
                      id, task.getBoolean("completed"));
        }    throw new IllegalArgumentException(
       "either description or completion status must be specified");
    }

That’s almost it — the only remaining task is to implement support for SSE, so clients can be notified when any of the existing tasks change or when tasks are added or deleted.

Implementing Server-Sent Events

Coherence provides a rich, powerful event model: Applications can observe server-side events, which are raised on the member that owns the entry that triggered the event, and client-side events, which allow you to observe the events happening across the cluster.

What’s needed is to provide an SSE endpoint at /api/tasks/events that the REST API clients can use to register for a stream of change events to update their local state whenever the set of tasks in the TaskRepository changes. All the clients should receive the same stream of events after the registration. The application leverages SSE broadcast support in JAX-RS to accomplish that:

@Inject
private TaskRepository tasks;@Inject
private Cluster cluster;@Context
private Sse sse;
private SseBroadcaster broadcaster;@PostConstruct
void createBroadcaster()
    {
    this.broadcaster = sse.newBroadcaster();    // TODO: convert Coherence events to SSE events
    }@GET
@Path("events")
@Produces(MediaType.SERVER_SENT_EVENTS)
public void registerEventListener(@Context SseEventSink eventSink)
    {
    broadcaster.register(eventSink);    Member member = cluster.getLocalMember();
    eventSink.send(sse.newEvent("begin", member.toString()));
    }

Once this code is added to the JAX-RS resource, the clients will be able to register with the SSE broadcaster. However, while the clients can register to observe the SSE events, the application isn’t actually listening to Coherence events, or sending any SSE events yet.

To fix the first issue and start listening to Coherence events, we need to register an event listener with the TaskRepository within the createBroadcaster method above:

@PostConstruct
void createBroadcaster()
    {
    this.broadcaster = sse.newBroadcaster();    tasks.addListener(
            tasks.listener()
                 .onInsert(this::sendInsert)
                 .onUpdate(this::sendUpdate)
                 .onRemove(this::sendDelete)
                 .build());
    }

Each of the event handlers that we registered above will be called with a Task that caused the event as an argument whenever the repository changes.

The SSE broadcaster expects us to publish OutboundSseEvent instances with the event name and the relevant data as the JSON payload, so we need to convert the Task we received into the OutboundSseEvent and publish it:

private void sendInsert(Task task)
    {
    broadcaster.broadcast(createEvent("insert", task));
    }private void sendUpdate(Task task)
    {
    broadcaster.broadcast(createEvent("update", task));
    }private void sendDelete(Task task)
    {
    broadcaster.broadcast(createEvent("delete", task));
    }private OutboundSseEvent createEvent(String name, Task task)
    {
    return sse.newEventBuilder()
                .name(name)
                .data(Task.class, task)
                .mediaType(APPLICATION_JSON_TYPE)
                .build();
    }

That concludes the REST API implementation. Let’s move on to the gRPC API.

Implementing gRPC endpoints

As ubiquitous as REST is, it is not the most efficient, or the simplest way to access remote services. To provide an alternate way of accessing tasks managed by our service, we will also create a gRPC facade for it.

To accomplish that we will use Helidon gRPC framework:

<dependency>
  <groupId>io.helidon.microprofile.grpc</groupId>
  <artifactId>helidon-microprofile-grpc-server</artifactId>
</dependency>

This allows us to implement gRPC service the same way we’ve implemented our REST service: via an annotated Java class.

Jakarta gRPC
Helidon gRPC is on its way to become an official Jakarta EE specification, via recently introduced Jakarta RPC project, led by yours truly.

@Grpc(name = "examples.json.ToDoList")
@GrpcMarshaller("jsonb")
@ApplicationScoped
public class ToDoListGrpcApiJson
    {
    @Inject
    private ToDoListService api;
    
    @Inject
    protected TaskRepository tasks;    
    
    // TBD    
}

As you can see, our gRPC service is just a CDI bean with a @Grpc and @GrpcMarshaller annotations, which define the service name clients should use to connect to it, and the marshaller to use (JSONB in this case), respectively.

Just like with our REST facade, most of the functionality can be implemented by simply delegating to the injected ToDoListService instance:

@Unary
public Task createTask(String description)
    {
    return api.createTask(description);
    }@ServerStreaming
public Stream<Task> getAllTasks()
    {
    return api.getTasks(null).stream();
    }@ServerStreaming
public Stream<Task> getTasks(boolean completed)
    {
    return api.getTasks(completed).stream();
    }@Unary
public Task findTask(String id)
    {
    return api.findTask(id);
    }@Unary
public Task deleteTask(String id)
    {
    return api.deleteTask(id);
    }@Unary
public boolean deleteCompletedTasks()
    {
    return api.deleteCompletedTasks();
    }

In a similar fashion, event streaming can be implemented by simply wiring gRPC @ServerStreaming methods to a repository listener:

@ServerStreaming
public void onInsert(StreamObserver<Task> observer)
    {
    tasks.addListener(
            tasks.listener()
                 .onInsert(observer::onNext)
                 .build());
    }@ServerStreaming
public void onUpdate(StreamObserver<Task> observer)
    {
    tasks.addListener(
            tasks.listener()
                 .onUpdate(observer::onNext)
                 .build());
    }@ServerStreaming
public void onRemove(StreamObserver<Task> observer)
    {
    tasks.addListener(
            tasks.listener()
                 .onRemove(observer::onNext)
                 .build());
    }

In this case there is no need to convert received Task instances to a wrapper event type — we simply stream them back via three different event channels, one for each event type.

Finally, in order to implement methods that allow us to update description and completion status, we need to define request messages that encapsulate task identifier and the value to update, as gRPC methods only accept a single argument:

@Unary
public Task updateDescription(UpdateDescriptionRequest request)
    {
    return api.updateDescription(request.id, request.description);
    }@Unary
public Task updateCompletionStatus(
                  UpdateCompletionStatusRequest request)
    {
    return api.updateCompletionStatus(
                    request.id, request.completed);
    }// ---- request messages -------------------------------public static class UpdateDescriptionRequest
    {
    public String id;
    public String description;
    }public static class UpdateCompletionStatusRequest
    {
    public String id;
    public boolean completed;
    }

That’s pretty much it for our gRPC facade implementation. Time to wrap it up by implementing the final, GraphQL facade.

Implementing GraphQL endpoints

GraphQL is an increasingly popular way to access data from all kinds of data sources, so our amazing To Do List service needs to support it.

To accomplish that, we will leverage Helidon’s implementation of Eclipse MicroProfile GraphQL specification. Let’s add a dependency on it to our pom.xml:

<dependency>
  <groupId>io.helidon.microprofile.graphql</groupId>
  <artifactId>helidon-microprofile-graphql-server</artifactId>
</dependency>

Just like with our REST and gRPC implementations, GraphQL facade is nothing more than a thin wrapper around our ToDoListService:

@GraphQLApi
@ApplicationScoped
public class ToDoListGraphQLApi
    {
    @Inject
    private ToDoListService api;
    
    //----- API methods -------------------------------
    @Mutation
    @Description("Create a task with the given description")
    public Task createTask(@Name("description") @NonNull 
                           String description)
        {
        return api.createTask(description);
        }    @Query
    @Description("Query tasks")
    public Collection<Task> getTasks(@Name("completed") 
                                     Boolean completed)
        {
        return api.getTasks(completed);
        }    @Query
    @Description("Find a given task using the task id")
    public Task findTask(@Name("id") @NonNull String id) 
        throws NotFoundException
        {
        return api.findTask(id);
        }    @Mutation
    @Description("Delete a task and return its details")
    public Task deleteTask(@Name("id") @NonNull String id) 
        throws NotFoundException
        {
        return api.deleteTask(id);
        }    @Mutation
    @Description("Remove all completed tasks")
    public boolean deleteCompletedTasks()
        {
        return api.deleteCompletedTasks();
        }    @Mutation
    @Description("Update task description")
    public Task updateDescription(
            @Name("id") @NonNull String id,
            @Name("description") @NonNull String description) 
        throws NotFoundException
        {
        return api.updateDescription(id, description);
        }    @Mutation
    @Description("Update task completion status")
    public Task updateCompletionStatus(
            @Name("id") @NonNull String id,  
            @Name("completed") boolean completed) 
        throws NotFoundException
        {
        return api.updateCompletionStatus(id, completed);
        }
    }

At the moment Eclipse MicroProfile GraphQL specification doesn’t support events, so we will not provide support for them in our GraphQL service facade either.

We have implemented a simple task management service with three API facades: REST, gRPC and GraphQL. We have also discussed various Coherence CE features along the way.

But does it work?

Running the server

The easiest way to run the server is to simply run it within your IDE of choice, which in my case is IntelliJ IDEA (see Figure 2):

Figure 2. Server run configuration in IntelliJ IDEA

Once you have the run configuration defined, click the Run button and Helidon will bootstrap both its own web server, which will serve the REST API, and a Coherence CE cluster member that will be used to store the data.

Wait…What!? That is correct: Coherence CE is not a server that needs to be started, unlike most data stores you may already be familiar with from relational databases, such as Oracle Database and MySQL, or NoSQL key-value data stores such as Mongo and Redis. It is a Java library, which can be easily embedded into any Java application! (It can even be embedded into Node.js applications using GraalVM.)

That’s not to say that there is nothing that needs to be started. Coherence CE (the library) provides a set of services that need to be started in order for it to form the cluster and manage data, for example, the Cluster service and a PartitionedService. However, because these services are essentially just Java classes that implement a com.tangosol.util.Service interface, they can be easily started programmatically within the application.

That’s exactly what happens here.

The Helidon io.helidon.microprofile.cdi.Main class will simply configure logging and bootstrap the CDI container. The CDI container will then look for all the available CDI extensions, and it will discover both the Helidon Web Server extension and the Coherence CE extension, among others. Both of these will observe an @Initialized event that will be fired by the CDI container once it’s ready for use and will then start the Helidon Web Server and the Coherence CE server, respectively.

This architecture provides two significant benefits:

It allows you to reduce complexity by effectively combining an application serverand a data store into a single stateful application server. This makes the architecture much simpler, because there are fewer elements to provision, scale, monitor, and manage. This tends to work quite well for simple applications and services. That is what I mean when I talk about the combination of Helidon and Coherence CE as a stateful microservices platform.
The architecture makes it easy to debug the code that executes within Coherence CE, such as entry processors, especially now that the actual, fully documented source code for Coherence CE is available on GitHub and on Maven Central.

Now, that doesn’t mean you always have to run the application that way. You can certainly choose to split the application server and the data store using Coherence CE roles and then manage them separately, but this is truly a deployment-time decision, not a development-time one. You can develop the application, run it within the IDE as a single process for testing and debugging, and even package it into a single Docker image. Then later you can choose to run separate app server and storage tiers by changing how you deploy the application to Kubernetes.

You can also run different microservices in a single Coherence CE cluster, or in multiple clusters, or even as a quasi-monolith by running all the services in every cluster member. The possibilities are endless, and the best option depends on your particular use case.

However, it is hard to overstate how much simpler the typical developer workflow is by simply embedding the data store into the app. Try it and you’ll see.

Go back to actually running the To Do List server. If you clicked on the Run button earlier, you should see log output similar to the following towards the end:

2020.07.30 05:55:53 INFO io.helidon.microprofile.server.ServerCdiExtension Thread[main,5,main]: Server started on http://localhost:7001 (and all other host addresses) in 12215 milliseconds (since JVM startup).

That output means the Helidon Web Server was successfully started and can be accessed on port 7001. You should also see the following a bit higher up in the log:

2020.07.30 05:55:53 CONFIG org.glassfish.jersey.server.ApplicationHandler Thread[main,5,main]: Jersey application initialized.
Root Resource Classes:
  com.oracle.coherence.examples.todo.server.rest.ToDoListRestApi

That output means the JAX-RS resource was discovered and deployed by Helidon.

Finally, you should be able to scroll a bit higher up and find the following:

2020.07.30 05:55:52 INFO coherence Thread[Logger@9258732 20.06,3,main]: (thread=DefaultCacheServer, member=1, up=10.634): 
Services
  (
  // omitted for brevity
  )
Started DefaultCacheServer...

That output means the Coherence CE server was started successfully and the application can actually store data in Coherence.

Much more information is logged by both Helidon and Coherence CE, so feel free to explore. However, these three sections in the log are critical, and if they are present, the To Do List service is ready to use.

Testing the REST API

I could’ve implemented automated tests for the REST API using JUnit and REST Assured, but for the time being, I use the curl tool to see if everything works as expected.

There’s no data in the application yet, so I’ll create some tasks using the POST endpoint:

$ curl -i -X POST -H "Content-Type: application/json" \
       -d '{"description": "Learn Coherence"}' \   
       http://localhost:7001/api/tasks
HTTP/1.1 204 No Content
$ curl -i -X POST -H "Content-Type: application/json" \
       -d '{"description": "Write an article"}' \ 
       http://localhost:7001/api/tasks
HTTP/1.1 204 No Content

Based on the HTTP response codes, that seems to be working. To check, run this command:

$ curl -i -X GET -H "Accept: application/json" \ 
       http://localhost:7001/api/tasks
HTTP/1.1 200 OK
Content-Type: application/json[{"completed":false,"createdAt":1596105616507,"description":"Learn Coherence","id":"9c6d9a"}
,{"completed":false,"createdAt":1596105656378,"description":"Write an article","id":"a3f764"}]

That output looks good. Can this application complete a task?

$ curl -i -X PUT -H "Content-Type: application/json" \
       -d '{"completed": true}' \ 
       http://localhost:7001/api/tasks/a3f764
HTTP/1.1 200 OK
Content-Type: application/json{"completed":true,"createdAt":1596105656378,"description":"Write an article","id":"a3f764"}

Awesome. The output shows that worked as well.

Conclusion

The article created a To Do List application using Coherence CE and Helidon, and it appears to be working correctly. I’ll leave the deletion of completed tasks and the deletion of tasks by ID as an exercise for readers, but you get the idea: you can store, update, delete, and retrieve tasks from Coherence CE using the REST, gRPC and GraphQL APIs developed earlier.

Although everything is working fine as long as you have the server up and running, you’ll notice that if you shut down the server you will lose all the data.

This is expected, because you haven’t configured Coherence CE to persist data to disk yet. So at the moment, data is stored only in memory and it’s lost if you restart the cluster, which currently has a single member. (I know: It’s not much of a “cluster” at this point, is it?)

One way to fix that is by starting more members, which will automatically enable high-availability mode and create backups of the tasks. Unfortunately, I can’t do that at the moment. Any additional members would fail to start because of TCP bind exceptions, because Helidon will attempt to start the web server on the same port (7001). That problem will go away in the final article of this series, which deploys the application to Kubernetes.

You can also prevent data loss on restart by enabling disk persistence, which can be accomplished by passing the -Dcoherence.distributed.peristence.mode=active system property on the command line, but I wouldn’t worry about it for now. It is faster to start the server with persistence disabled, and it’s usually simpler to start testing with a clean slate.

It is now time to build a decent looking front end for the application, so nobody will have to use curl to manage tasks ever again. However, I’ll leave that for the next article in this series.