Endpoint Manager by ITarian (EM) for stand alone installation. Basic features described on official site https://dm.comodo.com/
ITarian on-premise solution is distributed as set of docker images and can be run with docker. Therefore it is main requirement to have operation system with support and pre-installed docker
To make it works together for numerous docker-container it is used docker-compose tool
It allows easily setup all components together and maintain already run environment
All required action from system administrator of on-premise solution is:
Scheme of docker-compose usage:
Docker-compose supports stand alone configuration for single server. It is applicable for limited count of endpoints (up to 1000).
For scalable solution to manage multiple servers and horizontally scalable and reliable system need to use more complicated docker management system like a Kubernetes
At the moment support of kubernetes for on-premise solution is under the development and not supported
Minimum 2 servers for docker-compose configuration
Hardware requirements / recommendations (1000 endpoints)
By statistics: 1 endpoint produces 0.015 requests per second
It means that we can handle about 65 sequential requests from different endpoints per seconds.
Server could handle 50-100 connections simultaneously.
Therefore average endpoint count that can be handled is 50 * 65 > 3000.
ITSM-server
8 Cores Cpu | 8 Gb Ram | 100 Gb Hdd
Xmpp-server
4 Cores Cpu | 4 Gb Ram | 40 Gb Hdd
On-premise installation consists of multiple services and components which are communicated between themselves via network
Besides public ports most services expose specific ports to internal network which is closed to external world. These ports could be exposed just for debug purposes, but by default all service ports are closed including databases, message brokers and microservices which are the part of all system.
ITSM system designed for most restricted environments with corporate networks which has almost fully closed network
Therefore it only expose 443 port as main secure channel
Port 80 used only for convenient redirects as most popular default web port for each domain
For these purposes it was discovered also 443 port as default for xmpp connections to XMPP-server, but with difference to use TCP traffic instead of HTTP
Summarize it we need to have open 443 port on firewall as a minimum requirement. Port 80 also is recommended to open as it might confusing users and reduce usability
General tests held on Ubuntu Desktop and Ubuntu Server (Ubuntu 16.04.4 LTS).
Other versions of ubuntu not tested, but should work starting from Ubuntu 14 (maybe even Ubuntu 12)
Another versions of linux also should work (Debian, CentOS, etc). Only difference is how to install docker. All scripts below prepared and tested on Ubuntu 16.
For docker-compose configuration it doesn't matter which hostname is specified for each server.
Itsm requires several domain names which should be resolved by different components.
Expected that there are exists domain name and it could be resolved everywhere. Otherwise we have to specify each required subdomain on every endpoint according to infrastructure.
Basic dns domain should be set by customer. But there are few requirements for existing domains / subdomains.
List of required domains:
Base domain is just a pointer for all another subdomains.
ITSM-server IP 10.0.5.1 XMPP-server IP 10.0.5.2 Assume we have itsm domain on-prem.company.local on IP 10.0.5.1 (itsm-server). It means that base domain is company.local (doesn't matter which ip it has. This entry not used in the system). Next subdomains must be related to base domain. Rmm-domain - rmm-api.company.local (IP 10.0.5.1 same as itsm-server) Patch-Management-domain - plugins-api.company.local (IP 10.0.5.1 same as itsm-server) Audit-log-domain - auditlogs-api.company.local (IP 10.0.5.1 same as itsm-server) Download-domain - dl.company.local (IP 10.0.5.1 same as itsm-server) Xmpp-domain - xmpp.company.local (IP 10.0.5.2 XMPP-server) RealtimeDeviceCommunication-API-domain - rtdc-api.company.local (IP 10.0.5.1 same as itsm-server) RealtimeDeviceCommunication-Relay-domain - rtdc-relay-01.company.local (IP 10.0.5.1 same as itsm-server) BulkInstallationPackage-domain - bip.company.local (IP 10.0.5.1 same as itsm-server)
We recommend to have wildcard certificate. You may to have certificates for each subdomain specified above.
Currently required to have wildcard ssl certificate for itsm-server *.company.local and the same certificate or specific one for XMPP xmpp.company.local.
If you don’t have real domain and trusted SSL certificate you can generate self-signed certificates by yourself.
Note: endpoints couldn’t work with self-signed certificate on XMPP. In this case you have never get green online status on itsm-server for endpoints also remote control will not work.
For minimal configuration it is required to have set of certificates and keys for each server (itsm, xmpp)
Place valid ssl certificate and key into /opt/itsm/web/certs under the names cert.crt and cert.key.
Note: private key must be without passphrase as web server could not work with those
# create directory
sudo mkdir -p /opt/itsm/web/certs
# copy prepared certificate and key to destination
cp /path/to/your/certificate.crt /opt/itsm/web/certs/cert.crt
cp /path/to/your/certificate.key /opt/itsm/web/certs/cert.key
If you don’t have valid certificates. It is possible to issue self-signed certificate key-pair. But in this case you need to allow unsecured access in the browser and some features will be dropped. Next commands create self-signed certificates:
sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /opt/itsm/web/certs/cert.key -out /opt/itsm/web/certs/cert.crt
NOTE: your certificate CN (common name) domain should be the same as ITSM_DOMAIN variable value specified in .env
Place valid pem certificate for domain specified in XMPP_DOMAIN into /opt/ejabberd/certs
Filename should be in following pattern {XMPP_DOMAIN}.pem
NOTE:
sudo mkdir -p /opt/ejabberd/certs
cat cert.key cert.crt chain.crt root.crt > your.domain.pem
sudo mv your.domain.pem /opt/ejabberd/certs/
Prerequisites: install docker and docker-compose
Login to remote server
ssh username@ip-or-hostname
Get installation script (for ubuntu)
wget http://get-compose.on-premise.itarian.com/install-docker-compose.sh
Make file executable
chmod +x install-docker-compose.sh
Run script
sudo ./install-docker-compose.sh
Setup local user permissions
sudo usermod -a -G docker $USER
Logout from current session and login again to apply local user group changes
exit
ssh username@ip-or-hostname
Perform docker login
Note: your must have created account on CAM Comodo Accounts Management and order AEP license
docker login registry.on-premise.itarian.com
Enter CAM login and CAM password
Extra server setup
Only for itsm server it need to tune system settings:
sudo sysctl -w vm.max_map_count=262144
echo vm.max_map_count=262144 | sudo tee -a /etc/sysctl.conf
Logging Configurations
In order to apply log rotation and limiting the size of logs add below command under /etc/logrotate.conf
:
/var/lib/docker/containers/*/*.log {
rotate 5
copytruncate
missingok
notifempty
compress
maxsize 200M
daily
}
Or create file /etc/docker/daemon.json
with below commands:{
"log-driver": "json-file",
"log-opts": {
"max-size": "200m",
"max-file": "5",
"compress": "true"
}
}
Above configurations will rotate logs large than 200m and compress rotated logs. The system will keep original and 4 rotated log files. Configurations can be changed according to needs.Get docker-compose.yml and configure settings
Create and navigate to itsm dir
mkdir ~/itsm
cd ~/itsm
Get docker-compose.yml for specific server
for itsm server
wget https://get-compose.on-premise.itarian.com/version/release/registry-docker-compose.yml -O docker-compose.yml
for XMPP-server
wget https://get-compose.on-premise.itarian.com/version/release/registry-ejabberd-docker-compose.yml -O docker-compose.yml
create file with name .env and fill it according to your server requirements
ITSM_DOMAIN=on-premise.itsm.local
ITSM_TURN_SERVERS=ip of turn server
ITSM_XMPP_HOST=xmpp.itsm.local
ITSM_XMPP_IP=ip of xmpp server
ITSM_WEB_HOST=same as ITSM_DOMAIN on-premise.itsm.local
ITSM_WEB_IP=ip of this host
Where:
ITSM_DOMAIN - domain name which must be the same as the certificate domain used in setup
ITSM_TURN_SERVERS - list of ips where turn server is running separated by comma or space ( if turn servers has been setup )
ITSM_XMPP_HOST - domain for xmpp server ( if xmpp has been setup )
ITSM_XMPP_IP - ip for host specified in ITSM_XMPP_HOST if dns record cannot be resolved ( if xmpp has been setup without dns )
ITSM_WEB_HOST - domain name which used by rmm microservices and points to itsm-server.
ITSM_WEB_IP - ip for host specified in ITSM_WEB_HOST if dns record cannot be resolved
XMPP_DOMAIN=itsm.local
ITSM_WEB_HOST=on-premise.itsm.local
ITSM_WEB_IP=ip of itsm server
Prepare and save SSL certificates
Prepare ASCII encoded certificate files:
Compose and save certificate files
Save cert.crt and cert.key
sudo mkdir -p /opt/itsm/web/certs
cat cert.crt chain.crt | sudo tee /opt/itsm/web/certs/cert.crt
sudo cp cert.key /opt/itsm/web/certs/cert.key
Save single certificate bundle under the name specified in XMPP_DOMAIN and .pem extension
sudo mkdir -p /opt/ejabberd/certs
cat cert.key cert.crt chain.crt root.crt | sudo tee /opt/ejabberd/certs/itsm.local.pem
Starting up and basic management
Starting system
cd ~/itsm
docker-compose up -d
Stopping system
docker-compose down
View logs / debugging
docker-compose logs
To perform upgrade it need to go on next steps:
Go to itsm directory which contains docker-compose.yml and .env files
cd ~/itsm
Stop the system
docker-compose down
Get latest docker-compose.yml file for regarding to your server type
for itsm server
wget https://get-compose.on-premise.itarian.com/version/release/registry-docker-compose-with-turn.yml -O docker-compose.yml
for XMPP-server
wget https://get-compose.on-premise.itarian.com/version/release/registry-ejabberd-docker-compose.yml -O docker-compose.yml
Run the system again
docker-compose up -d
After complete installation need to setup SMTP to have ability to receive email from ITSM server
Problem: ITSM server not responding
Suggestion: Check docker logs for service "web"
Command
docker-compose logs -f web --tail 20
Output should not have constantly appended rows like
nginx: [emerg] host not found in upstream "service_name" in /etc/nginx/conf.d/website.conf:277
In case you find those rows, it mean that your service_name is not started
Check container logs with that service_name
docker-compose logs -f --tail 20 service_name
and check all containers that have not status Running or Exit 0
Problem: any docker command (example: docker ps) not works from your current user (Error like permission denied).
Suggestion: Check your current user belongs to group docker
Command
groups
Should contains docker
If user not belongs to group docker run this command
sudo usermod -a -G docker $USER
After that need to logout and login again with current user
Problem: any docker command (example: docker ps) not works. (Error like docker daemon is not running).
Suggestion: Check docker daemon status
Command
sudo service docker status
Output should have Active: active (running)
If output is Active: inactive (dead)
Start docker daemon with command
sudo service docker start
Problem: at least one container from list docker-compose ps has state Exit 1
Suggestion: Try to start this container.
Command
docker start <container name from column Name>
If problem persist - need to view container logs
Command
docker logs <container name from column Name> --tail 100
Output need to send to ITSM team for analyzing.
Problem: server data consumes lot of disk space (/var/lib/docker/overlay2)
Suggestion: Try to cleanup old docker images.
Every on-premise update pull several new images and keep both old and new ones
Command
docker image prune -af
Problem: server data consumes lot of disk space (/opt/itsm/elk)
Suggestion: Try to cleanup old elasticsearch indexes
On-premise provided with built-in elasticsearch management system kibana
Follow kibana setup guide to setup it
Remove indexes with kibana GUI
Problem: EM CCS clients updates consume lot of internet traffic
Suggestion: Setup local antivirus database updates
On-premise provided with av-db offline package
Follow antivirus local updates guide to setup CCS update preferences