Backend development is a large part of the work we are doing at High Output Ventures. Over time, we have accumulated a number of tools and conventions that we find ourselves repeatedly employing in every new project. I will be walking you through a typical backend development environment setup at High Output Ventures (HOV).
package.json
{
"name": "backend-environment",
"version": "0.0.0",
"description": "A typical backend development environment setup.",
"main": "build/index.js",
"scripts": {
"test": "exit -1"
},
"repository": {
"type": "git",
"url": "git+https://github.com/Proto-Garage/backend-environment.git"
},
"author": "High Output Ventures",
"license": "ISC",
"bugs": {
"url": "<https://github.com/Proto-Garage/backend-environment/issues>"
},
"homepage": "<https://github.com/Proto-Garage/backend-environment#readme>"
}
Most of the backend projects at HOV are written in Typescript and run on top of NodeJS. Just like any NodeJS project, we start by running npm init
. This will create a package.json
file that contains all the basic information in a NodeJS project.
Typescript
NodeJS does not execute Typescript code directly. Typescript codes must first be transpiled into Javascript before NodeJS can execute them. To transpile Typescript code into Javascript, we use the official typescript
package. We also use ts-node
to do the transpilation step at runtime during the development process. This runtime Typescript transpilation is particularly useful for running tests quickly and without the need for an intermediate build step. For path mapping, we use tscpaths
at compile-time and tsconfig-paths
at runtime.
npm install --save-dev ts-node tsconfig-paths tscpaths typescript
tsconfig.json
Every Typescript project must come with a tsconfig.json
file. By carefully choosing the options we include in this file, we can customize the Typescript build process so that it fits perfectly with our setup. To simplify the debugging process, we enable the sourceMap
option. By setting the --enable-source-maps
option in NodeJS, every time a runtime error occurs, the stack trace will point us to the correct line number in the source file. We also make use of the path mappingfeature to get rid of the long relative paths in import statements. The Typescript files are kept in ./src
and the build files are sent to ./build
.
{
"compilerOptions": {
"target": "es2017",
"module": "commonjs",
"sourceMap": true,
"outDir": "./build",
"strict": true,
"strictNullChecks": true,
"strictFunctionTypes": true,
"strictPropertyInitialization": true,
"noImplicitAny": true,
"removeComments": true,
"noImplicitThis": true,
"alwaysStrict": true,
"noUnusedLocals": true,
"noUnusedParameters": true,
"moduleResolution": "node",
"esModuleInterop": true,
"lib": ["es2020"],
"baseUrl": ".",
"paths": {
"@library/*": ["src/library/*"],
}
},
"include": ["src/**/*"]
}
Production Dependencies
We use a large number of production dependencies, but the ones that are almost always used in every project are luxon
and ramda
. luxon
is an alternative to the more famous moment
package. We prefer luxon
over moment
because of its significantly better API. We use luxon
for everything that involves date and time manipulation. For utility functions, we use ramda
. We fell in love with ramda
because of its strong adherence to functional programming principles. Other production dependencies that we commonly use are: ms
, uuid
, redis
, mongoose
, and koa
.
npm install luxon ramda
npm install --save-dev @types/luxon @types/ramda
Linting
To catch potential issues in the code, we use eslint
for linting. We also use a number of additional plugins which enables us to use eslint
together with Typescript.
npm install --save-dev @typescript-eslint/eslint-plugin @typescript-eslint/parser eslint eslint-config-airbnb-base eslint-import-resolver-typescript eslint-plugin-import
.eslintrc
We use the Airbnb eslint
config as the base config. We then add rules on top as we see fit.
{
"parser": "@typescript-eslint/parser",
"plugins": ["@typescript-eslint", "import"],
"extends": [
"airbnb-base",
"plugin:@typescript-eslint/recommended",
"plugin:import/typescript"
],
"settings": {
"import/resolver": {
"typescript": {}
}
},
"env": {
"node": true,
"mocha": true
},
"rules": {
"max-len": ["error", 128],
"no-console": "off"
}
}
Tests
We primarily use mocha
to run our tests. We have also experimented with alternative test runners such as ava
and cucumberjs
. To generate random inputs, we use chance
and to implement stubs, we use sinon
. To measure the test coverage, we use nyc
. All the test files are kept at ./test
.
npm install --save-dev mocha chance sinon nyc @types/mocha @types/chance @types/sinon
Tests are usually divided into two categories: unit tests
and integration tests
. Unit tests are aimed at testing individual components while integration tests are aimed at testing multiple components together to check if they work properly as a whole. We also keep a collection of custom helper functions for repetitive tasks such as generating test objects and setting up test environments.
tsconfig.json
The test
directory contains its own tsconfig.json
file. This local tsconfig.json
file is basically an extension to the tsconfig.json
file in the root directory. Here, we add options that are unique to the test environment.
{
"extends": "../tsconfig.json",
"compilerOptions": {
"baseUrl": ".",
"paths": {
"@library/*": ["../src/library/*"],
"@helpers/*": ["./helpers/*"]
}
},
"include": [
"./**/*",
"../src/**/*",
],
}
.mocharc.json
mocha
options can be set in the .mocharc.json
file which is usually located in the root directory. In our setup, we register ts-node
so that we can directly run tests without needing an intermediate transpilation process. We also register tsconfig-paths
to resolve path mapping at runtime.
{
"colors": true,
"exit": true,
"recursive": true,
"bail": true,
"timeout": 5000,
"require": ["ts-node/register/transpile-only", "tsconfig-paths/register"],
"spec": "test/**/*.spec.ts"
}
.nycrc
To measure the test coverage, we use istanbul
and it’s command-line interface nyc
. By running nyc
in every build, we ensure that there is enough test coverage at all times. The coverage threshold configuration is set depending on the needs of the project.
{
"extends": "@istanbuljs/nyc-config-typescript",
"all": true,
"check-coverage": true,
"lines": 75,
"statements": 75,
"branches": 75,
"functions": 75
}
Docker
Most of the applications that we built run inside Docker containers. These containers are then deployed to either Docker Swarm or Kubernetes. Using containers makes it easier to package and deploy our projects.
Dockerfile
Our typical Docker image is built on top of an alpine
base image. alpine
containers are known to be lightweight and minimalist.
FROM node:12.18.2-alpine
WORKDIR /srv/node
COPY ./build /srv/node/build
COPY ./package.json /srv/node/package.json
COPY ./node_modules /srv/node/node_modules
CMD npm start
docker-compose.yml
To deploy third-party services such redis
and mongodb
in the local development environment, we use docker-compose
. This ensures that all of our developers are using the same set of services with exactly the same versions.
version: '3.2'
services:
redis:
image: redis:5.0.5-alpine
ports:
- 6379:6379
mongo:
image: mongo:4.1.5-xenial
ports:
- 27017:27017
Scripts
Custom scripts may be added to the package.json
file. In our setup, these custom scripts and their functions are listed as follows:
build
– Transpiles the Typescript source files and resolves thepath mappings
.clean
– Removes the./build
directory.lint
– Executeseslint
and reports any linting errors.lint:fix
– Executes eslint and attempts to fix any linting errors automatically.start
– Starts the NodeJS application. This assumes that thebuild
script was previously executed.test
– Executes all tests.typecheck
– Checks if there are existing Typescript type errors.
{
...
"scripts": {
"build": "npm run clean && tsc && tscpaths -p tsconfig.json -s ./src -o ./build",
"clean": "rimraf /build",
"lint": "eslint . --ext .ts",
"lint:fix": "eslint . --ext .ts --fix",
"start": "node --enable-source-maps build/index.js",
"test": "cross-env NODE_ENV=test TS_NODE_PROJECT=tsconfig.json TS_NODE_FILES=true nyc --reporter=lcov --reporter=text-summary mocha",
"typecheck": "tsc --noEmit"
},
...
}
Closing Thoughts
There is an infinite number of ways to set up a backend development environment. What works for our team may not work for others. What’s important is that we choose the tools and conventions that we think are the best fit given the technical requirements of our projects. It is also important that we continually evolve this setup as new trends appear in the industry. An example project following this setup can be found here.
More from
Engineering
Importance of Design QA workflow
Marvin Fernandez, Engineering Manager
SQA questions to ponder
Joy Mesina, QA
Writing a Software Test Plan
Ron Labra, Software Quality Analyst