How to create new Digicor services

Pre-requisites

java jdk 8
mvn archetype:generate...
git clone
[email protected]:digicor/java-libs/digicor-service-archetype.git

Design functionality

define user story and use cases
- use Given, When, Then style - simple and short (https://martinfowler.com/bliki/GivenWhenThen.html)
define front-end (modules/views)
- what user sees and how can interact with it (GUI look and feel)
- what API GUI needs from the new service as well as from other services (define required interactions)
define backend services. We promote using micro service architecture where each service has only one responsibility. As a result interaction with several services may be required to fulfil one use case. Define:
- where is data from front-end stored (which backend service is responsible for providing/managing the data)
- how is interaction delivered to other user
- which APIs the service should publish
- which events the service consumes and produces
  - APIs and events are typically negotiated between developers of producers and consumers

The above analysis is required to define new service and its interaction (GUI, REST API, consumed events, produced events).

For Java developers you can use our Maven archetype - the maven archetype provides a way to create a new service. New service includes all important features you can build on and adapt it for your needs
- https://maven.apache.org/
- https://maven.apache.org/guides/introduction/introduction-to-archetypes.html
For developers in other languages than Java, all steps must be done manually. In detail, following interactions have to be taken into account (built in the above generated Java service, otherwise) when developing the service
- REST API for public communication
- Eventuate as an Event Store for inter service communication via events http://eventuate.io/
  - Eventuate has REST API to produce new events and STOMP API to consume events.
- Docker to make an image for our application https://www.docker.com/
- Docker Compose to locally test the service images https://docs.docker.com/compose/
- Kubernetes to deploy and manage service images in cloud environment https://kubernetes.io/
- Amazon Web Services to host our cloud https://aws.amazon.com/

Configure maven credentials for Digicor

Get credentials for nexus.digicor-platform.eu - Access of partners to AWS GitLab
add following lines to your .m2/settings.xml (more details at http://maven.apache.org/settings.html) and change your Nexus credentials there

http://maven.apache.org/SETTINGS/1.0.0"

xmlns:xsi=""

xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0

">

digicor-maven

your-username

your-password

archetype

your-username

your-password

digicor

archetype

Digicor NEXUS maven

digicor-maven

Digicor NEXUS maven

digicor

New service from maven archetype

Our maven archetype can be found in git

digicor-service-archetype.git

Now you can simply build new service using the maven archetype -

just fill your data marked in silver below

mvn archetype:generate -U \

-DarchetypeArtifactId=digicor-service-archetype \

-DarchetypeGroupId=cz.certicon.digicor \

-DarchetypeVersion=1.9 \

-DgroupId=cz.certicon.digicor \

-DartifactId=service-name \

-Dversion=1.0-SNAPSHOT \

-Dpackage=cz.certicon.digicor.servicename \

-DserviceName=ServiceName \

-DservicePort=8080 \

-Dauthor= \

-Ddate=2017-11-27 \

-DinteractiveMode=false

archetypeVersion - use the latest version from pom.xml in our git (21.9. 2018 - current latest version is 1.8)
- digicor-service-archetype/blob/master/pom.xml
groupID - identification of the developing partner (company). Reversed domain name is recommended.
artifactId - name of maven artifact, reflected also in the local service folder name and service name in cloud
package - generated default package for your service source code
serviceName - service class name. This name will affect names of generated classes e.g. ${serviceName}Handler.class, ${serviceName}HandlerEvent.class, etc.
servicePort - port your service will listen at. This must be unique far all services in the cloud - uniqueness will be defined later and conflicting ports will be changed

Maven archetype generates service application and its events in the following folder structure.

Apart from java clases it contains build pipeline configuration file

.gitlab-ci.yml

kubernetes configuration file

service-name-app/kubernetes/service-name.yaml

and docker file for the service to configure docker image with runnable script

service-name-app/Dockerfile

service-name-app/service-name-app.sh

Generated Java source code

It is necessary to understand general concepts described in Development guidelines to be able to follow. Also for understanding details of following steps see http://eventuate.io/gettingstarted.html

If your service needs to publish some event you need to define it in service-name-events module

if you do not need it just remove it

Otherwise:

define command in commands package
You can create new event instance by command only. Command classes must extend an aggregate-specific Command superinterface. (see http://eventuate.io/docs/javav2/java-commands.html for details)
define event in events package.
For each event type implement event Class (see http://eventuate.io/docs/javav2/java-events.html for details) . The class describes the event - it may hold data required for event consumers. i.e. ProductionPlanCreated event holds data for new ProductionPlan.
create ServiceNameAggregate class in the domain package defining what events are created when a given command is processed in the process method (see http://eventuate.io/docs/javav2/java-aggregates.html for details)
configuration package contains spring configuration for publisher/consumer to configure connection to event store for given Aggregate
service package contains service which works with event store.
Services use AggregateRepository class, parametrized by the given aggregate and the command, that enables saving and updating Aggregate data. (for details see http://eventuate.io/docs/javav2/java-services.html)
- you can save/find/update entities in event store there

Service functionality will be implemented in service-name-app

handler package contains handlers that contain a code to be executed when consuming events (from this service defined in service-name-events)
- it can be used if service need to persist its own data to event store - local service database is not persisted, if service restarts local data is lost
hateos contains basic GET /service-name/{id} method configuration
BackendConfiguration spring configuration to configure events, local view service and handler for events
RestAPIConfiguration spring configuration to configure Swagger and Eventuate for production use
test package contains test spring configuration RestAPITestConfiguration uses embedded event store and application tests for spring context load, /health endpoint and swagger-ui
other sources contains application.properties for docker and application_local.properties for direct run from jar

Developing and testing

Java JDK 8

Docker compose is not meant to replace unit tests in your service. Please write unit tests. There are some technologies to consider when your are embracing TDD in your development effort.

https://site.mockito.org/

https://junit.org/junit4/

https://spring.io/

https://www.baeldung.com/injecting-mocks-in-spring

There is plenty of tutorials on the Internet. If you would like, you can find inspiration in source code of:

java-services/company

java-services/tool-store

java-services/tender

aws-authorizer

jwt-authorization

Build and run tests locally (build it before each commit and push)

cd service-name

mvn clean install

Build and run service with Eventuate locally

If you need to run more services or service with eventuate interaction you can use docker compose. However, using docker-compose is not meant to replace tests in your code. It is meant only for experimenting with eventuate. If you are developing service which is listening for events it is advised to write tests which will "send" the event without need of eventuate infrastructure. Running tests is much faster than sending events by sending REST requests to running services. Some of the services require much more than eventuate to run. For example company service requires ElasticSearch, AWS S3 and AWS Cognito. Company service will be returning Internal Error to every request when ElasticSearch and AWS Cognito is missing. It is not our intention to be able to create Digicor on localhost also it is almost impossible, because of resource requirements (more than 60 GB of RAM).

Now your generated code could be build (JDK 1.8 is recommended, there may be some issues when Java 1.9 is used):

cd service-name

mvn clean install

service-name-events
service-name-app

Login to docker-private.digicor-platform.eu

$ docker login docker-private.digicor-platform.eu -u your-nexus-user-name -p your-nexus-password

Login Succeeded

Build your own docker image

cd service-name/service-name-app

mvn package

docker build -t service-name .

you can find docker image in your local docker repo

docker images

you can run docker image

docker run service-name

docker-compose

set DOCKER_HOST_IP environment variable -

http://eventuate.io/docs/usingdocker.html

clone repository -

docker-images/local-cluster

This configuration of Eventuate infrastructure can send events with payload around 1 000 000 characters. Conventional settings could send only 21 000 characters (default settings only 1000). Be aware that sending events with huge payload is not good practice in Event Sourcing. Huge payload usually means flaws in design of events and interactions between services.

#192.168.99.100 is IP where docker host is running

#bash

export DOCKER_HOST_IP=192.168.99.100

#win cmd

set DOCKER_HOST_IP 192.168.99.100

git clone :digicor/docker-images/local-cluster.git

cd local-cluster

docker-compose -f docker-compose.yml up -d

If you are on Windows and have some issues with volume mount try to set the following environment variable:

set COMPOSE_CONVERT_WINDOWS_PATHS=1

Deploy your service into digicor-platform.eu

Repository for your service will be create upon request. Please contact us via contact page on this site and provide us a name of your service. Your repository will be created in java-services.
You could push your service into the repository.
After push, the GitLab CI pipeline will start automatically. Pipeline is defined in the file .gitlab-ci.yml. If you have any questions about CI, please do not hesitated to contact us.
Service will be build automatically including docker images.
The deployment step of the pipeline has to be manually confirmed in GitLab -> your-project -> CI/CD -> pipeline. This will deploy your service into Kubernetes cluster. The Kubernetes configuration files are in kubernetes/your-service-name.yaml. There should be no need to change this configuration file. When you need to change this file, please let us know.

Currently only project owner is allowed to push into master branch of his project. If you want to participate on project where you are not a owner you have to create new branch and then create a merge request.

Start with empty repository

git clone :digicor/java-services/production-plan.git

mvn archetype:generate ...

git add -A

git commit -m "message"

git push

Start with existing folder

mvn archetype:generate ...

cd your-service-name

git init

git remote add origin :digicor/java-services/your-service-name.git

git add -A

git commit -m "Initial commit message"

git push -u origin master

DIGICOR