Ansible initial server setup

With this article we see how to simplify the initial setup of a new physical and virtual server with the use of Ansible.

Ansible environment installation

In order to continue it is necessary to have a linux computer where Ansible is present.

To install Ansible you can use your own distribution package or use the pip command and create a virtualenv where you can install everything at the latest version or the desired version without changing the system libraries.

prerequisites installation

First we install, if not present, python3 and pip:

• on RedHat systems >=8: sudo dnf install -y python3 python3-pip
• while on RedHat systems <8:  sudo yum install -y python3 python3-pip (it needs the epel repository)
• finally on Debian/Ubuntu systems: sudo install -y python3 python3-pip python3-venv

Virtual environment venv for Python

Now as a user we create our python virtual environment:

cd $HOME && mkdir venv && python3 -m venv venv/ansible

We activate our python virtual environment with:

source venv/ansible/bin/activate

At this point with this shell we have a new python environment where we can install whatever we want without interfering with the system one.

From now on we consider to be in virtualenv or on a python environment where we can install libraries with pip.

Ansible, lint and molecule installation

We update to the latest pip version with:

pip install setuptools && pip install --upgrade pip

Than install ansible with:

pip install ansible

So we can test it with:

ansible -v

and check the ansible_facts in localhost with:

ansible localhost -m setup

To check the yaml syntax and the Ansible specific one we install the parsers:

pip install yamllint ansible-lint

To test Ansible rules automatically via Docker we install molecule:

pip install molecule docker molecule[docker]

Ansible Role: initial server setup

We then move on to the main project, which is the creation of an Ansible role for the initial configuration.

cd $HOME && mkdir roles
cd roles
ansible-galaxy role init eniocarboni.test
cd eniocarboni.test
molecule init scenario -r eniocarboni.test -d docker

The first thing to do is edit the metadata in the meta/main.yml file by adding or editing inside the galaxy_info tag:

galaxy_info:
  role_name: test
  namespace: eniocarboni
  author: Your Name
  company: Yours
  ...

In addition, at least the license and the compatibility of the role with the various distributions must be changed:

license: GPL-3.0
platforms:
  - name: Ubuntu
    versions:
      - all

At this point we can already test our role with molecule that will automatically use Docker with its default image (quay.io/centos/centos:stream8):

molecule test

since we have both yamllint and ansible-lint installed let’s configure them for use with molecules:

cat <<EOF >>molecule/default/molecule.yml
lint: |
  set -e
  yamllint .
  ansible-lint
EOF

cat <<EOF >.ansible-lint
---
exclude_paths:
  - molecule
EOF

Re-running the test we will see that molecule also uses yamllint and ansible-lint:

molecule test

Of course we can also launch commands directly without molecules:

yamllint .
ansible-lint

test on a real vm

To test on a physical or virtual server we need to have access via ssh with a user who has access to root via sudo or to root directly.

So what we will need is:

• the name of the server or its ip: testserver;
• the user to login: user;
• the playbook file: playbook.yml;
• the inventory file: inventory_hosts

So just do:

mkdir -p tests/mytest
echo "testserver" >tests/mytest/inventory_hosts
cat <<EOF >tests/mytest/playbook.yml
---
- hosts: all
  become: true
  roles:
    - eniocarboni.test
EOF

To test directly in the testserver server just run:

ansible-playbook -u root -i tests/mytest/inventory_hosts tests/mytest/playbook.yml

Task to perform

At this point we need to add the tasks to be done on the server and put them in the tasks/main.yml file such as:

• installing new packages;
• host name change;
• time zone change;
• setting of the main language;
• …

Let’s briefly see how to do it:

cat <<EOF >tasks/main.yml
---
- import_tasks: update_packages.yml
- import_tasks: hostname.yml
- import_tasks: update_timezone.yml
- import_tasks: update_locale.yml
EOF
for f in update_packages hostname update_timezone update_locale
do
  echo "---" >tasks/$f.yml
done

We then complete the first update_packages.yml file with:

cat <<EOF >>tasks/update_packages.yml
- name: update apt cache
  ansible.builtin.apt:
    update_cache: yes
    cache_valid_time: 3600
  when: ansible_pkg_mgr == "apt"
- name: update yum cache
  ansible.builtin.yum:
    update_cache: yes
  when: ansible_pkg_mgr == "yum"
- name: update dnf cache
  ansible.builtin.dnf:
    update_cache: yes
  when: ansible_pkg_mgr == "dnf"
- name: Install new packages
  ansible.builtin.package: "name={{ item }} state=latest" # noqa package-latest
  with_items: "{{ packages | default([]) }}"
EOF

So just put the default variables such as packages in the defaults/main.yml file:

cat <<EOF >>defaults/main.yml
packages:
  - wget
  - openssh-server
EOF

Finally we can test the first tasks with molecule via docker or on our test server and verify that everything is working correctly:

molecule test

INFO     default scenario test matrix: dependency, lint, cleanup, destroy, syntax, create, prepare, converge, idempotence, side_effect, verify, cleanup, destroy
INFO     Performing prerun...
...
INFO     Running default > dependency
WARNING  Skipping, missing the requirements file.
WARNING  Skipping, missing the requirements file.
INFO     Running default > lint
WARNING  Loading custom .yamllint config file, this extends our internal yamllint config.
WARNING  Failed to discover lintable files using git: fatal: non è un repository Git (né lo è alcun genitore fino al punto di mount /)
Mi fermo al limite del filesystem (l'opzione GIT_DISCOVERY_ACROSS_FILESYSTEM non è impostata).
INFO     Running default > cleanup
WARNING  Skipping, cleanup playbook not configured.
INFO     Running default > destroy
INFO     Sanity checks: 'docker'

PLAY [Destroy] *****************************************************************

TASK [Destroy molecule instance(s)] ********************************************
changed: [localhost] => (item=instance)

TASK [Wait for instance(s) deletion to complete] *******************************
FAILED - RETRYING: [localhost]: Wait for instance(s) deletion to complete (300 retries left).
ok: [localhost] => (item=instance)

TASK [Delete docker networks(s)] ***********************************************

PLAY RECAP *********************************************************************
localhost                  : ok=2    changed=1    unreachable=0    failed=0    skipped=1    rescued=0    ignored=0

INFO     Running default > syntax

playbook: /home/enio/Progetti/ansible/eniocarboni.test/molecule/default/converge.yml
INFO     Running default > create
PLAY [Create] ******************************************************************

TASK [Log into a Docker registry] **********************************************
skipping: [localhost] => (item=None) 
skipping: [localhost]

TASK [Check presence of custom Dockerfiles] ************************************
ok: [localhost] => (item={'image': 'quay.io/centos/centos:stream8', 'name': 'instance', 'pre_build_image': True})
...
...
TASK [eniocarboni.test : update apt cache] *************************************
skipping: [instance]

TASK [eniocarboni.test : update yum cache] *************************************
skipping: [instance]

TASK [eniocarboni.test : update dnf cache] *************************************
ok: [instance]

TASK [eniocarboni.test : Install new packages] *********************************
changed: [instance] => (item=wget)
changed: [instance] => (item=openssh-server)

PLAY RECAP *********************************************************************
instance                   : ok=3    changed=1    unreachable=0    failed=0    skipped=2    rescued=0    ignored=0

INFO     Running default > idempotence
...
...
PLAY RECAP *********************************************************************
instance                   : ok=3    changed=0    unreachable=0    failed=0    skipped=2    rescued=0    ignored=0

INFO     Idempotence completed successfully.
INFO     Running default > side_effect
WARNING  Skipping, side effect playbook not configured.
INFO     Running default > verify
INFO     Running Ansible Verifier

PLAY [Verify] ******************************************************************

TASK [Example assertion] *******************************************************
ok: [instance] => {
    "changed": false,
    "msg": "All assertions passed"
}

PLAY RECAP *********************************************************************
instance                   : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

INFO     Verifier completed successfully.
INFO     Running default > cleanup
WARNING  Skipping, cleanup playbook not configured.
INFO     Running default > destroy

PLAY [Destroy] *****************************************************************

TASK [Destroy molecule instance(s)] ********************************************
changed: [localhost] => (item=instance)

TASK [Wait for instance(s) deletion to complete] *******************************
FAILED - RETRYING: [localhost]: Wait for instance(s) deletion to complete (300 retries left).
changed: [localhost] => (item=instance)

TASK [Delete docker networks(s)] ***********************************************

PLAY RECAP *********************************************************************
localhost                  : ok=2    changed=2    unreachable=0    failed=0    skipped=1    rescued=0    ignored=0

INFO     Pruning extra files from scenario ephemeral directory

ansible-playbook -u root -i tests/mytest/inventory_hosts tests/mytest/playbook.yml

At this point we should continue with all the other tasks that we have left empty and possibly add others, an operation that I will not do in this article so as not to dwell too much.

Ansible role: eniocarboni.server_initial_setup

For those interested can use my role on Ansible Galaxy eniocarboni.server_initial_setup by installing it with:

ansible-galaxy install eniocarboni.server_initial_setup

and checking or cloning the code on GitHub where you can also find all the documentation on how to use it.

This role is tested in various versions of Ubuntu and RedHat systems or equivalent.

The tasks it performs right now are (and are all configurable):

• update package cache;
• updating packages to the latest version;
• installing new packages;
• package for automatic update via cron based on the distribution;
• screen or tmux installation;
• hostname change;
• timezone change;
• locales change;
• adding or deleting ssh public keys;
• added sysctl parameters;
• firewall setup (ufw on Debian/Ubuntu, Firewalld on RedHat systems or iptables)

References

• Ansible: other from quoll or by ufficial site;
• Python;
• pip;
• epel repository;
• Docker;
• molecule;
• yaml;
• yamllint;
• ansible-lint;
• ssh;
• Ubuntu;
• Debian;
• RedHat;
• eniocarboni.server_initial_setup on Ansible Galaxy and on GitHub