Itential Automation Platform

On this page:

Installation

This guide defines the minimum requirements to have a functioning production or development instance of the Itential Automation Platform (IAP). Instructions on how to install core software and configure IAP properties are provided.

Requirements

This section includes required and recommended installation requirements.

Physical Servers vs Virtual Machines

The IAP platform and all of its components support both physical servers and virtual machines. Virtual machines (VM) are recommended because they are less expensive and easier to manage as a solution. Moreover, performance appears to be comparable, assuming the VM hardware specs are guaranteed and not shared with other virtual machines.

Production

For production environments, all IAP components should be installed on their own individual servers to properly support high availability (HA).

Lab and Development

For non-production environments the specs may be lowered to provide lower cost options. However, keep in mind that depending on usage, these specs will need to be adjusted. For highly active developer environments with a great number of custom applications, increased CPU, memory, and disk space will be required.

IAP Server

The following table describes required IAP production server components.

Component Requirement Production ENV Lab / Test ENV Development ENV
CPU 64-bit x86 CPU cores. Min: 8 / Rec: 16 Min: 4 / Rec: 8 Min: 2 / Rec: 4
RAM Memory Min: 32GB / Rec: 64GB Min: 16GB / Rec: 32GB Min: 4GB / Rec: 8GB
Disk Total
/var/log/pronghorn
/opt/pronghorn
/var/log/redis
250 GB
100 GB
100 GB
50 GB
125 GB
50 GB
50 GB
25 GB
65 GB
25 GB
25 GB
15 GB

The following hardware specs assume an IAP Server with a few custom applications. In situations where a number of custom applications are heavily utilized, or additional IAP backup storage is needed then the CPU, RAM, and disk space allocations will need to be increased.

MongoDB Server

The following table describes MongoDB server resources that are required by IAP in a production environment.

Component Requirement Production ENV Lab / Test ENV Development ENV
CPU 64-bit x86 CPU cores. Min: 8 / Rec: 16 Min: 4 / Rec: 8 Min: 2 / Rec: 4
RAM Memory Min: 64GB / Rec: 128GB Min: 16GB / Rec: 32GB Min: 4GB / Rec: 8GB
Disk Total
/var/log/mongo
/data
1 TB
150 GB
850 GB
500 GB
100 GB
400 GB
250 GB
50 GB
200 GB

Redis Shared-Token Server

The following table describes Redis server resources that are required by IAP in a production environment using multiple IAP servers and shared-tokens.

Component Requirement Production ENV Lab / Test ENV Development ENV
CPU 64-bit x86 CPU cores. Min: 4 / Rec: 8 Min: 2 / Rec: 4 Min: 1 / Rec: 2
RAM Memory Min: 4GB / Rec: 8GB Min: 2GB / Rec: 4GB Min: 1GB / Rec: 2GB
Disk Total
/var/log/redis
50 GB
50 GB
25 GB
25 GB
10 GB
10 GB

RabbitMQ Server

The following table describes RabbitMQ server resources that are required by IAP in a production environment.

Component Requirement Production ENV Lab / Test ENV Development ENV
CPU 64-bit x86 CPU cores. Min: 4 / Rec: 8 Min: 2 / Rec: 4 Min: 1 / Rec: 2
RAM Memory Min: 4GB / Rec: 8GB Min: 2GB / Rec: 4GB Min: 1GB / Rec: 2GB
Disk Total
/var/log/rabbitmq
50 GB
50 GB
25 GB
25 GB
10 GB
10 GB

Alternative Lab and Development Environments

As an alternative solution a single larger VM may be used to run all of the IAP platform components. A lab environment may include MongoDB, Redis, RabbitMQ, and IAP on a single server.

Component Requirement Lab / Test ENV Development ENV
CPU 64-bit x86 CPU cores. Min: 8 / Rec: 16 Min: 4 / Rec: 8
RAM Memory Min: 16GB / Rec: 24GB Min: 8GB / Rec: 16GB
Disk Total
/var/log/pronghorn
/opt/pronghorn
/var/log/redis
/var/log/rabbitmq
/var/log/mongo
/data
650 GB
50 GB
150 GB
25 GB
50 GB
75 GB
300 GB
425 GB
25 GB
100 GB
25 GB
25 GB
50 GB
200 GB

Host Operating System

IAP is supported on the following operating systems.

OS Release Production Development
CentOS 64-bit CentOS 7 x x
RHEL 64-bit RHEL 7 x x

OS Packages

Install the following operating system packages on each IAP server.

  • coreutils
  • openssl
  • ntp
  • nscd
  • epel-release
  • gcc-c++
  • make

Security

The following security related packages are recommended.

  • iptables
  • auditd
  • selinux-policy
  • selinux-policy-targeted
  • selinux-policy-mls
  • policycoreutils
  • libselinux
  • libselinux-utils
  • setools-console

Operational

For operational troubleshooting, the following packages are recommended.

  • telnet
  • bind-utils
  • sysstat
  • tcpdump
  • dos2unix
  • curl
  • wget
  • zip
  • unzip
  • gzip
  • man

Required Software

IAP requires the following software be installed.

Software
Redis
MongoDB
Node.js
Node.js Package Manager
RabbitMQ

Note: Itential Automation Platform (IAP) is compatible with several third-party software products for network operating environments. All third-party software version compatibility is documented in Release Notes. To get the most up-to-date requirements for any third-party software, including open source, first identify which IAP release you’re using and then refer to the respective release note.

Installing Dependencies

Install the prerequisite software: Node.js, Redis, MongoDB, and RabbitMQ. Links to additional documentation resources have been provided below.

Node.js must be installed on the same server as IAP. Redis, MongoDB, and RabbitMQ should be installed on separate servers from IAP in a production deployment, but alternatively can be installed on the same server as IAP for an all-in-one type development deployment. If a shared token Redis instance exists, it can be used rather than a second Redis instance. See the High Availability Architecture section for production deployment options.

Note: The following instructions assume component installation on a server running CentOS 7. The Redis, MongoDB, and RabbitMQ steps will work either directly on the IAP server or a standalone server, unless stated otherwise.

Node.js

Please refer to the NodeSource README.md for details on Node.js installation.

Post Installation

The version of Node.js that IAP requires may be found in the Release Notes of the the IAP release.

Verify that Node.js is installed and available from the PATH environment variable.

$ command -v node
/bin/node

Check which version of Node.js is installed.

$ node -v
v12.16.2

Node.js Package Manager

The Node.js Package Manager tool, npm, is installed along with Node.js. However, a minimum version is required.

Post Installation

Verify that npm is installed and available from the PATH environment variable.

$ command -v npm
/bin/npm

Check which version of npm is installed.

$ npm -v
6.14.4

If the version of npm is not greater than or equal to version 6, then upgrade npm with the following command.

npm i -g npm@6.14.4

Redis

Please refer to the Redis documentation for details on Redis installation.

Post Installation

If Redis is running on a different server than IAP, the Redis process may bind to the localhost. If this occurs then Redis and IAP may not be able to communicate.

The following are example steps for allowing Redis to communicate outside of localhost.

  1. Edit /etc/redis.conf on the Redis server.

  2. Comment out bind 127.0.0.1 so that it becomes #bind 127.0.0.1.

  3. Restart Redis.

    sudo systemctl restart redis

Note: The instructions open the Redis TCP port externally, so security hardening should be considered.

Configuration

Apply the following recommended Redis configurations to a production or development environment. These recommendations are relevant where Redis is running in a virtual machine. However, many of these configurations may also apply to a bare metal installation.

Max Memory

By default, Redis will consume up to 80% of the memory available on the platform. Set the upper limit to 20% of the available memory so IAP may consume the remaining 80%.

Change the # maxmemory <bytes> value in the /etc/redis.conf file.

Restart Redis to allow the /etc/redis.conf changes to go into effect.

sudo systemctl restart redis

Note: Further Redis configuration for a production environment can be found in the Redis Configuration of the IAP Admin Guide.

RabbitMQ

Please refer to the RabbitMQ installation guide for details on RabbitMQ installation.

Note: Verify the installed RabbitMQ version matches the version specified in the Release Notes dependency section for the relevant IAP version.

Post Installation

Check that the RabbitMQ service is running.

$ systemctl status rabbitmq-server

    rabbitmq-server.service - RabbitMQ broker
    Loaded: loaded (/usr/lib/systemd/system/rabbitmq-server.service; enabled; vendor preset: disabled)
    Active: active (running) since Thu 2020-01-02 17:38:38 UTC; 5s ago

Note: RabbitMQ TCP port 5672 is exposed externally by default. Hardening RabbitMQ security should be considered.

MongoDB

Please refer to the official MongoDB installation instructions.

Post Installation

To verify a successful MongoDB installation:

  1. Start the MongoDB service.

    sudo service mongod start
  2. Show the operational status of the mongod service.

    $ sudo service mongod status
    Redirecting to /bin/systemctl status mongod.service
    ● mongod.service - High-performance, schema-free document-oriented database Loaded: loaded (/usr/lib/systemd/system/mongod.service; enabled; vendor preset: disabled)
    Active: active (running) since Fri 2018-11-09 16:12:57 UTC; 3 weeks 0 days ago
    Docs: https://docs.mongodb.org/manual
    Main PID: 7587 (mongod)
    CGroup: /system.slice/mongod.service
          !"7587 /usr/bin/mongod -f /etc/mongod.conf
    Nov 09 16:12:56 localhost.localdomain systemd[1]: Starting High-performance,schema-free document-oriented database...
    Nov 09 16:12:56 localhost.localdomain mongod[7583]: about to fork child process, waiting until server is ready for connections.
    Nov 09 16:12:56 localhost.localdomain mongod[7583]: forked process: 7587
    Nov 09 16:12:57 localhost.localdomain mongod[7583]: child process started successfully, parent exiting
    Nov 09 16:12:57 localhost.localdomain systemd[1]: Started High-performance, schema-free document-oriented database.

Refer to the MongoDB Configuration section of the IAP Admin Guide to review configuration considerations for a production environment.

Note: The network settings specified in the mongod.conf file may need verification. This file is set to 127.0.0.1 but may need to be changed to 0.0.0.0. For more information on adapting the config settings for this file, refer to the MongoDB documentation.

Standalone Server

If running MongoDB on a server other than the IAP server, perform the following steps so that IAP will be able to reach it.

  1. Edit /etc/mongod.conf on the MongoDB server.

  2. Change the line starting with bindIp: 127.0.0.1 so that it becomes bindIp: 0.0.0.0.

  3. Restart MongoDB.

    sudo systemctl restart mongod

Note: MongoDB security hardening is recommended as the previous steps open the default MongoDB TCP port 27071 externally.

Template Builder Dependencies

To install Template Builder dependencies:

  1. Install Pip for Python 2.

    sudo yum -y install python-pip
  2. Run the following commands. Once TextFSM and Jinja2 are available, the template parser will be able to function properly.

    pip install textfsm
    pip install jinja2

Installing IAP

Perform the following steps to install the IAP software onto a CentOS 7 server. Be sure the latest version of the IAP package has been copied to the server. You can acquire the link for this build from the portal.

Installation

Perform the following steps to install IAP onto the server.

  1. Transfer the IAP binary package to the server.

  2. Change to directory, cd, where the package is stored.

  3. Run the install binary as a privileged user. The default installation directory is /opt/pronghorn/current.

    sudo bash itential-<build-id>_version.linux.x86_64.bin -i

Note: Change <build-id> in the installer binary above to match the version being installed.

Configuration

Upon successful installation, a default IAP configuration file will be created at /opt/pronghorn/current/properties.json. This file will initially contain a default IAP configuration that may be edited to match the environment.

  1. Edit /opt/pronghorn/current/properties.json.

  2. Change all references of mongodb://mongo:27017 to either the hostname/IP address of your MongoDB server or localhost if hosted on the IAP server.

    Example Options

      # Option 1) Hosted locally on IAP server.
      mongodb://localhost:27017
    
      # Option 2) Hosted on a standalone MongoDB server.
      mongodb://mongodb.test.com:27017
    
      # Option 3) MongoDB replica set.
      mongodb://mongo01.test.com:27017,mongo02.test.com:27017,mongo03.test.com:27017?replicaSet=rs0
  3. Change all references of "host": "redis" to either the hostname or IP address of your Redis server or localhost if hosted on the IAP server.

    Examples

      # Option 1) Hosted locally on IAP server.
      "host": "localhost"
    
      # Option 2) Hosted on a standalone Redis server.
      "host": "redis.test.com"

DB Migration

A migration script must be manually run to import properties.json configuration into the MongoDB database. After the migration script is run once, the properties.json file will only be referenced by IAP for the MongoDB connection properties, with IAP referencing the database for all other configuration properties.

Note: Any desired changes to properties.json should be made before proceeding.

  1. Enter the directory containing the IAP migration script.

    cd /opt/pronghorn/current/node_modules/@itential/pronghorn-core/migration_scripts
  2. Run the migration script. Key/value pairs after --userInputs can be changed to match the RabbitMQ deployment. Examples:

    # Option 1) RabbitMQ installed on IAP server with default port.
    node migratePropertiesToDatabase.js --userInputs protocol=amqp port=5672 hosts="localhost"
    
    # Option 2) RabbitMQ installed on standalone server.
    node migratePropertiesToDatabase.js --userInputs protocol=amqp port=5672 hosts="rabbitmq.test.com"

Note: The properties.json file will be updated during the migration script run to only contain the MongoDB connection properties, with all other configuration parameters stored within the default Mongo database named pronghorn. The migration script creates a backup of the original properties.json file that will look similar to properties_b4b03d30-ad00-4f61-bd9e-7953968ef8c4.json.

Start IAP

  1. Start IAP and set to auto-start on server reboot.

      systemctl enable --now pronghorn
  2. Confirm IAP has started successfully.

      $ systemctl status pronghorn
    
      pronghorn.service - Itential Automation Platform Service
        Loaded: loaded (/etc/systemd/system/pronghorn.service; enabled; vendor preset: disabled)
        Active: active (running) since Thu 2020-01-02 20:37:43 UTC; 4min 29s ago
      Main PID: 30774 (Pronghorn core)

Note: The journalctl -u pronghorn -f command may be run to monitor IAP logs during startup or general usage.

Access IAP Web UI

IAP may be accessed by browsing to http://<SERVER_HOSTNAME>:3000/login with <SERVER_HOSTNAME> being either the hostname or IP address of the server that IAP is installed on. The default user is admin@pronghorn with password admin.

Note: Be sure to give the admin@pronghorn user access to all roles from the Settings->Authorization page in the IAP Web UI in order to access applications.

Password Encryption

Passwords stored within the properties.json IAP properties file should be encrypted. This can be achieved by using the encryption script that is included within the pronghorn-core package.

Note: Password encryption is one-way. Be sure all passwords are securely stored as retrieval is not possible.

Perform the following steps to run the encryption script and generate a password value.

  1. Navigate to the pronghorn-core node modules directory.

    cd /opt/pronghorn/current/node_modules/@itential/pronghorn-core
  2. Run the following command where mypassword is the password value that requires encryption.

    $ npm run encrypt mypassword
    Encrypted Password:
    $ENC8ef3972b5766e64a98df4b11d6d3221d82812e8caed3459e5a0d

Use the encrypted password value, beginning with $ENC, instead of the plain-text values that are inserted in the properties.json file.

AAA

IAP supports local group authorization and external authentication, authorization and auditing (AAA). Local groups may be created and roles may be assigned. Users which have previously logged in to IAP may be assigned to local groups.

IAP comes pre-configured with a Local AAA adapter that stores user accounts and passwords in a MongoDB database. The adapter works well with a quick-start application but should not be used in production environments.

IAP supports the following external authentications.

  • Active Directory
  • LDAP
  • Radius

Secure MongoDB Connectivity in Local AAA

Use this section to setup the Local AAA adapter in IAP using a password and SSL protected MongoDB.

Note: The information presented here assumes Mongo is not using authorization.

  1. Setup the Admin user.

    db.createUser(
    {
        "user":"admin",
        "pwd":"password",
        "roles":[
            {
                "role":"root",
                "db":"admin"
            },
            {
                "role":"userAdminAnyDatabase",
                "db":"admin"
            },
            {
                "role":"clusterMonitor",
                "db":"admin"
            },
            {
                "role":"dbOwner",
                "db":"LocalAAA"
            },
            {
                "role":"dbOwner",
                "db":"pronghorn"
            }
        ]
    }
    )
  2. Setup the Pronghorn user.

    db.createUser(
    {
        "user":"pronghorn",
        "pwd":"password",
        "roles":[
            {
                "role":"dbOwner",
                "db":"pronghorn"
            },
            {
                "role":"dbOwner",
                "db":"LocalAAA"
            },
            {
                "role":"clusterMonitor",
                "db":"admin"
            }
        ]
    }
    )
  3. Setup the local AAA user.

    db.createUser(
    {
        "user":"localaaa_user",
        "pwd":"pronghorn",
        "roles":[
            {
                "role":"dbOwner",
                "db":"LocalAAA"
            }
        ]
    }
    )
  4. Modify the mongod.conf file to turn on authorization.

    /etc/mongod.conf
    # network interfaces
    net:
    port: 27017
    bindIp: 0.0.0.0  
    # Listen to local interface only, comment to listen on all interfaces.
    security:
    authorization: enabled
  5. Modify the properties.json file.

    /opt/pronghorn/current/properties.json
    "id": "profile1",
    "mongoProps": {
        "credentials": {
        "dbAuth": true,
        "passwd": "password",
        "user": "pronghorn"
        },
        "db": "pronghorn",
        "url": "mongodb://127.0.0.1:27017"
    }
  6. Modify properties for the MongoDB adapter via IAP (navigate to Settings > Services > Adapters).

    "properties": {
        "id": "mongo",
        "properties": {
            "credentials": {
                "dbAuth": true,
                "passwd": "password",
                "user": "pronghorn"
            },
            "db": "pronghorn",
            "url": "mongodb://127.0.0.1:27017"
        },
  7. Modify Local AAA properties.

    Note: In this example, "pronghorn" was used for the password. This is consistent with how the Local AAA user was set in Step 3 above.

    "properties": {
        "id": "Local AAA",
        "type": "local_aaa",
        "properties": {
            "database": {
                "db": "LocalAAA",
                "url": "mongodb://127.0.0.1:27017",
                "credentials": {
                    "dbAuth": true,
                    "passwd": "pronghorn",
                    "user": "localaaa_user"
                }
            }
        },
        "brokers": [
            "aaa"
        ],
        "groups": []
    },
  8. Restart MongoDB.

    systemctl restart mongod
  9. Stop Pronghorn (Itential).

    systemctl stop pronghorn
  10. Start Pronghorn (Itential).

    systemctl start pronghorn
  11. Check status of Pronghorn (Itential).

    systemctl status pronghorn

Encrypt Passwords in Local AAA (Optional)

To encrypt passwords:

cd /opt/pronghorn/current/node_modules/@itential/pronghorn-core/utils
npm run encrypt.js

Alternately, you can use:

node encrypt.js

The encrypt tool will respond with a string that starts with $ENC. For example:

password:  $ENC93eb9439537ae34196db49409dda241c8282228cabd94098

pronghorn: $ENC93f88824437dfe5784c7570f99d72f1d81822788aad246995b

Replace the unencrypted password with the encrypted password.

Unencrypted

"passwd": "password"

Encrypted

"passwd": "$ENC93eb9439537ae34196db49409dda241c8282228cabd94098"

Additional Adapter Installation (Optional)

IAP uses network and OSS adapters to facilitate the integration of IAP with various systems and applications.

In order to add a new adapter to IAP, the adapter module must be installed under node_modules within the IAP installation directory. For example:

npm install @itentialopensource/adapter-db_mongo

Once the adapter module is installed, restart IAP and the adapter will be auto-discovered during IAP startup and become available when adding an adapter via the IAP UI.

The following steps will add an available adapter to IAP.

  1. Access the IAP Web UI.

  2. Browse to Settings -> Services.

  3. Select the plus button next to Adapters, fill in the desired name of the adapter and the type, and click the save icon.

  4. A skeleton JSON object of the Service Configuration for the adapter will be created and can be edited to match your environment.

    Service Configuration

  5. Save your final adapter Service Configuration. The adapter status can be confirmed from Settings -> System -> Adapters.

For more information, see the IAP Integrations section under the IAP Admin Guide.

Troubleshooting Hints

User Cannot Log In

The default user and password values are admin@pronghorn and admin.

User Cannot See Applications

Upon browsing to the IAP Web UI as the admin@pronghorn user for the first time, you will be presented with the following message:

You currently don't have access rights to any apps. If you think that's a mistake, please ask your Administrator for help.`

Be sure to add permissions to the admin group for your applications.

Authorization Manager

NSO Service Models Missing

Be sure to add the service models to the ncs.conf file load-path.

<load-path>
    <dir>./packages</dir>
    <dir>./yang</dir>
    <dir>${NCS_DIR}/etc/ncs</dir>

    <!-- To disable northbound snmp altogether -->
    <!-- comment out the path below -->
    <dir>${NCS_DIR}/etc/ncs/snmp</dir>
</load-path>

Invalid Credentials for Local AAA

If you are getting invalid credentials, the Local AAA user has not been setup properly, or you have the wrong credentials or parameters in your properties file for the Local AAA adapter.

Check to see if the user and password for Local AAA is valid:

mongo -u localaaa_user -p pronghorn LocalAAA

show users....

If you are logged in and can see the configured users, check the Local AAA adapter properties to verify they are set correctly. The same also applies to pronghorn if IAP does not come up; be sure the pronghorn user and password are valid in the properties.json file and in the adapter properties for Mongo.