Subscription

The core provides builtin features to subscribe both free and paid support services.

The Redis HASH key cluster/subscription holds the attributes of the subscription.

provider The subscription provider identifier. Possible values: nsent, nscom
system_id System subscription identifier (see Credential formats)
auth_token Authentication token for subscription APIs (see Credential formats)
collect_url (nsent only) Base URL of the my collect API the cluster sends heartbeat, inventory and backup to, e.g. https://my.nethesis.it/collect/api/systems. It is set only when the cluster holds native my credentials; while it is set, migrate-to-my does nothing
migrated_at (nsent only) RFC3339 UTC timestamp of the credential rotation. Set only when a legacy cluster is migrated, not on a fresh native registration
legacy_system_id, legacy_auth_token (nsent only) The pre-migration credential pair, preserved for audit / rollback when migrate-to-my rotates a legacy cluster to native my credentials
dartagnan_url (nscom only) Community subscription API base URL (default https://my.nethserver.com/api)
vpn_cert_cn X509 Common Name for server certificate validation
vpn_peer_host (default “sos.nethesis.it”)
vpn_peer_port (default “1194”)
support_user User name added to cluster-admin when a support session is started on the leader node. Password is set to a random UUID value. The account is removed when the support session is stopped

When a cluster has the Redis HASH key cluster/subscription some Systemd services and timers are started and enabled at boot. The unit check-subscription.service starts at boot and keep all the other unit status aligned with the Redis key value. This is a list of the services managed by check-subscription:

send-inventory (leader only) Run at night, send the cluster inventory to the provider
send-heartbeat (leader only) Run every 10 minutes to signal the cluster liveness
send-backup (leader only) Run at night, send the cluster encrypted backup to the provider
apply-updates (leader only) Run at night, apply core, modules and OS updates according to configuration in Redis key cluster/apply_updates. See also core updates

The subscription status and running services are checked when:

the node boots
the node joins the cluster
the cluster is restored from a cluster backup file
the cluster subscription is enabled or disabled
the cluster leader node changes

Credential formats

The format of system_id and auth_token depends on the provider:

nsent (Nethesis Enterprise, new my) — the customer pastes the full my token, of the form my_<public>.<secret>, which is stored as auth_token. set-subscription posts it to the my POST /systems/register API, which validates it and returns the system_key, stored as system_id, of the form NETH-XXXX-XXXX-XXXX-XXXX-XXXX-XXXX-XXXX-XXXX (NETH- followed by eight 4-hex-digit groups). A system_id is one-shot: once registered on my it cannot be re-registered, and a new registration attempt is rejected with HTTP 409 (system_already_subscribed).
nscom (Community) — auth_token is the community token; system_id is the machine UUID returned by dartagnan (my.nethserver.com/api/machine/info).
Legacy nsent (before migration) — clusters registered on the old my.nethesis.it used a UUID system_id and a hex auth_token. During the transition to the new my, migrate-to-my rotates the pair to native NETH-… / my_… credentials, stores collect_url and migrated_at, and preserves the old pair under legacy_system_id / legacy_auth_token.

Migration to the new my

migrate-to-my runs once, at the end of the core update. Until it succeeds, a legacy nsent cluster does not send heartbeat, inventory and backup to my.

If it fails, the Subscription page shows a Complete migration button. It runs the migrate-subscription cluster action, which fails if the cluster is still not migrated:

api-cli run migrate-subscription

Or run the script again as root on the leader node:

runagent -m cluster migrate-to-my

Check the result in the journal:

journalctl -t migrate-to-my

On success, collect_url and migrated_at are set:

redis-cli hgetall cluster/subscription

The command is safe to run more than once: it does nothing if the cluster is already migrated, is not nsent, or is not subscribed.

APIs

Cluster:

get-subscription
set-subscription

Node:

get-support-session
start-support-session
stop-support-session

Events

subscription-changed

Remote support

Support sessions are started/stopped by each cluster node separately. Relevant node actions are

start-support-session
stop-support-session

For example, it is possible to start/stop a support session for any node of the cluster. Run on the leader node this command:

api-cli run node/7/start-support-session

The action prints the session ID, required to ask support:

{"session_id": "811630de-67f4-5b6d-8b2f-7cc01f592b1b"}

As alternative, log in on the node and start manually the support service with this command:

systemctl start support
systemctl status support

When a node support session is started

support SSH key is added to the node /root/.ssh/authorized_keys file with an expiry-time= attribute. Only connections from the node itself are allowed with that key.
support credentials are enabled for cluster-admin access. The user name is defined in Redis. Type HGET cluster/subscription support_user to see its value, by default it is nethsupport. The password is set to the support session ID value. nethsupport can connect from localhost and a restricted set of IP addresses, coded in the support-clusteradminctl command.
an OpenVPN tunnel is established with the support server, allowing connections to sshd and cluster-admin. The support server address and port can be overridden in the Redis cluster/subscription key

When the support session is terminated

the OpenVPN tunnel is closed
cluster-admin access is removed
SSH key is removed

The support session is automatically terminated after 24 hours. To avoid automatic termination and allow it to run up to the maximum allowed duration of 7 days, execute this command on the relevant node:

systemctl stop support-expire.timer

After 7 days the session is terminated unconditionally.

Inspect the node session expiry with the get-support-session action. For example:

api-cli run node/7/get-support-session

OpenVPN support client

The Systemd unit support.service controls an OpenVPN client running in a container, within a separate network namespace where the TUN device is created. This network namespace is configured with nft to forward some TCP ports from the VPN tunnel to services of the node. For example

Forward VPN ports 981 to 127.0.0.1:22
Forward VPN ports 22, 9090, 80 and 443 to the respective port on 127.0.0.1

See nft.conf for the port forwarding rules.

Inventory format

The inventory format is compatible with NS7 inventory and merges data collected on the leader node with data collected by calls to the get-facts action of each module instance.

If a module implements the get-facts action, its output is merged in the inventory data, no matter the node hosting it.

This is an excerpt of get-facts data merged with user and group counters for the LDAP domain ad.example.org:

{
  "cluster_module_domain_table": [
    {
      "active_users": 65,
      "domain_id": "ad.example.org",
      "instance_of": "mail",
      "module_id": "mail8",
      "node_id": 5,
      "total_groups": 31,
      "total_users": 86
    },
    {
      "active_users": 65,
      "domain_id": "ad.example.org",
      "instance_of": "ejabberd",
      "module_id": "ejabberd1",
      "node_id": 5,
      "total_groups": 31,
      "total_users": 86
    },
    {
      "active_users": 41,
      "domain_id": "ad.example.org",
      "instance_of": "nextcloud",
      "module_id": "nextcloud1",
      "node_id": 1,
      "total_groups": 31,
      "total_users": 92
    },
    {
      "active_users": 65,
      "domain_id": "ad.example.org",
      "instance_of": "webtop",
      "module_id": "webtop7",
      "node_id": 5,
      "total_groups": 31,
      "total_users": 86
    },
    {
      "active_users": 0,
      "domain_id": null,
      "instance_of": "mattermost",
      "module_id": "mattermost1",
      "node_id": 1,
      "total_groups": null,
      "total_users": 116
    }
  ],
  "cluster_user_domains_table": [
    {
      "active_users": 65,
      "domain_id": "ad.example.org",
      "ejabberd_count": 1,
      "mail_count": 1,
      "nextcloud_count": 1,
      "total_groups": 31,
      "total_users": 86,
      "webtop_count": 1
    }
  ]
}

Notes:

The Mattermost instance mattermost1 is not bound to the LDAP domain, users are stored into the module database. Its get-facts action returns active_users and total_users counters with data stored inside its database.
The Nextcloud instance nextcloud1 implements get-facts. Its output overrides the active_users and total_users counters inherited from the ad.example.org user domain.
Instances mail8, ejabberd1 and webtop7 inherit counters from domain ad.example.org