User domains

Users and groups are stored in an LDAP database, served by one account provider module. Multiple modules can work together to serve the same LDAP database as replicas of it. An LDAP database represents an account domain.

A NS8 cluster can host multiple account domains from different implementations. It is possible to configure and connect external LDAP services, too. Supported LDAP schema are

  1. ad - Active Directory
  2. rfc2307

LDAP service discovery

A module can discover the list of available account domains with the agent.ldapproxy Python module. The following command dumps a list of parameters required to connect with an LDAP database on cluster node 1.

runagent python3 -magent.ldapproxy

Returned TCP endpoints are local (host is 127.0.0.1) and do not require TLS. The port number depends on the LDAP domain.

Those ports are held by the Ldapproxy module. It is a L4 proxy that relays the TCP connection to an LDAP backend server, enabling TLS and handling backend failures as needed.

If the LDAP client module runs in a Podman container with a private network, add the following arguments to the podman run command:

--network=slirp4netns:allow_host_loopback=true

Then replace 127.0.0.1 with the special 10.0.2.2 IP address, that is translated by Podman back to the loopback device, 127.0.0.1 on the root network namespace.

Python code example:

from agent.ldapproxy import Ldapproxy
lp = Ldapproxy()
domains = lp.get_domains_list()
print(domains)
domain = lp.get_domain("mydomain")
print(domain)

The module can handle the user domain configuration changes by defining an event handler. Create an executable script with path ${AGENT_INSTALL_DIR}/events/user-domain-changed/10handler. For instance:

mydomain="ad.example.org"

# Check if $mydomain is in the list of changed domains
if jq -e --arg d "$mydomain" '.domains | index($d)' >/dev/null; then
    systemctl --user reload mymodule.service
fi

List users and groups

Once LDAP connection parameters are retrieved with the agent.ldapproxy Python package, it is easy to get user and group listings with the agent.ldapclient package.

This is an excerpt from the cluster/list-domain-groups API implementation:

from agent.ldapproxy import Ldapproxy
from agent.ldapclient import Ldapclient

domparams = Ldapproxy().get_domain('mydom.test')
groups = Ldapclient.factory(**domparams).list_groups()

For complete examples see the API implementation of

  • cluster/list-domain-groups
  • cluster/list-domain-users
  • cluster/get-domain-user
  • cluster/get-domain-group

Hidden users and groups

Some users and/or groups can be hidden to UI and other applications.

Applications might need to build LDAP search filters to configure user and groups. The Ldapproxy library provides some methods that return filter strings that honor the user and group lists used by the core. For example:

from agent.ldapproxy import Ldapproxy
lp = Ldapproxy()
users_filter = lp.get_ldap_users_search_filter_clause("mydomain")
print(users_filter)
groups_filter = lp.get_ldap_groups_search_filter_clause("mydomain")
print(groups_filter)

Bind modules and account domains

If a module wants to use an account domain it must be granted API permissions. Add the accountconsumer role to the org.nethserver.authorizations label of the module image. For instance set

org.nethserver.authorizations=cluster:accountconsumer

The module can now execute a bind procedure, so the core is aware of existing relations between modules and account domains. When such relations are formally established the core can

  • limit/grant access to LDAP resources
  • show the relations in the web user interfaces

For example, a module that uses one domain at a time can unbind the old domain and bind the new one with a script like this:

import agent

ldap_user_domain = "dept1.example.org"

# Bind the new domain, overriding previous values (unbind)
agent.bind_user_domains([ldap_user_domain])

At any time, retrieve the list of domains currently bound:

import agent
rdb = agent.redis_connect(use_replica=True)
domlist = agent.get_bound_domain_list(rdb)

When the module or the domain is removed from the cluster, the relation cleanup occurs automatically.

If the module wants to be notified of any change to the relation between modules and user domains it can subscribe the module-domain-changed event. For instance, this is the payload of such event:

{
    "modules": ["mymodule1"],
    "domains": ["mydomain.test"]
}

The event payload contains a list of module and domains that were affected by the relation change. Modules and domains can be either added or removed: they are listed to ease the implementation of event handlers in both account provider and account client modules.

For instance, the following Python excerpt checks if the module domain was changed:

event = json.load(sys.stdin)
if not os.environ["LDAP_USER_DOMAIN"] in event["domains"]:
    sys.exit(0) # nothing to do if our domain is among affected domains

# Handle the event by some means, for example
# - rewrite some config file
# - reload some service running in a container

Single sign-on

A module can delegate user authentication to an OpenID Connect (OIDC) provider module. The core does not implement one: it relies on the minimal interface described here, and the ns8-idp module (Keycloak) is its reference implementation.

An OIDC provider serves a realm for each user domain, named after the domain, with the users and groups of its LDAP database. A module gets an OIDC client in the realm of a user domain it is bound to. The preferred_username claim of its tokens is the user name in the user domain, so the module can match OIDC logins with existing accounts.

OIDC provider discovery

A provider publishes the module/{MODULE_ID}/srv/http/oidc service key and raises the service-oidc-changed event when it changes. The key fields are:

  • host: the public host name of the provider
  • issuer_url_prefix: the OIDC issuer of a user domain is this prefix followed by the domain name, for example https://sso.example.org/realms/dp.example.org

The event payload carries the key fields, so listeners can tell whether the change concerns them, besides the key name and the provider module:

{
    "host": "sso.example.org",
    "issuer_url_prefix": "https://sso.example.org/realms/",
    "key": "module/idp1/srv/http/oidc",
    "module_id": "idp1",
    "module_uuid": "8d257122-0a7f-49c7-a620-08961a68cfa0"
}

Endpoints and keys of the issuer are found with the standard OIDC discovery document, {issuer}/.well-known/openid-configuration. To list the providers:

import agent
rdb = agent.redis_connect(use_replica=True)
providers = agent.list_service_providers(rdb, 'oidc', 'http')

OIDC client registration

A provider implements the register-client action, granted by its clientadm role. The client module declares it in its authorizations label, for example

org.nethserver.authorizations=idp@any:clientadm

The client module binds the user domain first, then calls the action of the chosen provider:

import agent
response = agent.tasks.run(agent_id="module/idp1", action="register-client", data={
    "domain": "dp.example.org",
    "redirect_uris": ["https://cloud.example.org/apps/user_oidc/code"],
    "post_logout_redirect_uris": ["https://cloud.example.org/"],
})
agent.assert_exp(response["exit_code"] == 0)
# {"client_id": "nextcloud1", "client_secret": "...", "realm": "dp.example.org",
#  "issuer": "https://sso.example.org/realms/dp.example.org"}

The provider must:

  • name the client after the calling module ID; an explicit module_id input attribute is accepted only from tasks without a calling user, like those of the cluster agent or api-cli
  • reject a domain not bound to the calling module
  • create the realm if missing, and bind itself to the user domain
  • update an existing client and return its current secret; the rotate_secret flag generates a new one
  • keep the enabled state of an existing client: a client disabled by an administrator stays disabled when its module registers it again
  • delete the client when its module is removed or unbound from the domain

Other input attributes:

  • redirect_uris: without redirect URIs, the client can only authenticate its own requests, for example the token introspection of a mail server
  • post_logout_redirect_uris: the module should pass the exact URI its application uses at logout. If the attribute is missing, the provider allows any URI of the origins of the redirect URIs, like https://cloud.example.org/*
  • web_origins: the allowed CORS origins
  • audience: client IDs added to the token audience. For example a webmail client adds the mail module client, because the mail server accepts only tokens that list it

Validation errors have the field and error attributes of the standard action validation output. The error values are module_id_required, module_id_not_allowed, caller_not_a_module and domain_not_bound_to_module. See also the register-client input schema.

A provider can restrict a realm to logins through a federated identity provider, like Microsoft Entra ID. Such a realm refuses LDAP passwords, but a module that shows its own login form still accepts them: in that case the module should offer exclusive SSO, sending its users straight to the provider, and document an emergency login path for when the provider is down. Protocols outside the browser, like IMAP and WebDAV, keep using LDAP passwords.

Import/Export users APIs

Samba and OpenLDAP core modules implement a uniform API to massively import and export users. To use that API modules must be authorized:

org.nethserver.authorizations=samba@any:domadm openldap@any:domadm

With domadm role, a client module can invoke the following actions and user-portal handlers:

  • export-users
  • import-users

This is an example CSV file with 9 fields, that can be used to feed import-users input:

dolores.slughorn3,Dolores Slughorn,Magic!123,dolores.slughorn3@hogwarts.example,aurors|hufflepuff|quidditch,true,false,true
cedric.macmillan4,Cedric Macmillan,Magic!123,cedric.macmillan4@hogwarts.example,hufflepuff|ministry|quidditch,false,true,false
pansy.dumbledore5,Pansy Dumbledore,Magic!123,pansy.dumbledore5@hogwarts.example,gryffindor|ravenclaw,true,false,false

Run this Bash command to import the CSV (non-existing groups are created on the fly):

cat file.csv | jq -R -s '
    split("\n")
    | map(select(length>0))
    | map(split(","))
    | {
        skip_existing: false,
        records:
            map({
            user: .[0],
            display_name: .[1],
            password: .[2],
            locked: (.[5] | test("(?i)^(1|true|yes)$")),
            groups: (.[4] | split("|") | map(ascii_downcase)),
            mail: .[3],
            must_change_password: (.[6] | test("(?i)^(1|true|yes)$")),
            no_password_expiration: (.[7] | test("(?i)^(1|true|yes)$"))
            })
        }
    ' | api-cli run module/samba1/import-users --data -

Actions/handlers for single users and groups are also authorized with role domadm:

  • add-user
  • add-group
  • alter-user
  • alter-group
  • remove-user
  • remove-group
  • list-users
  • list-groups

Password expiration warning

The node leader can send email notifications to users when their password is about to expire. The notification is available only for internal user domains.

This features has 2 requirements:

  • password aging must be enabled on the user domain provider
  • the cluster must be configured to send mail notifications using an internal or external SMTP server

A timer named password-warning.service runs daily on all nodes to check for expiring passwords, but it actually sends the email only on the leader node.

Configuration is saved inside Redis so it is available to all nodes:

  • cluster/password_warning/<domain>
  • cluster/password_warning/templates

See database schema for details.

User destination mail address

The notify-password-warning script obtains the destination address from the LDAP mail attribute of each user entry in OpenLDAP or Samba LDAP DB. The attribute is editable from cluster-admin UI and backend actions.

For user entries without mail LDAP attribute, the script checks the existence of a Mail application with a configured mail domain named after the user domain. If such mail domain exists in the cluster, the destination address is set to <user>@<user_domain>.

  • If that Mail application is also the one configured as SMTP server for notifications, the submission is internal and user_domain does not need to be resolvable via a public DNS MX record.
  • Otherwise, delivery to user_domain follows conventional SMTP rules and does require a public DNS MX record.

For example user john of user domain ad.example.org has no address in the mail LDAP attribute. A Mail application with matching domain ad.example.org is searched. If one is found the notification is sent to john@ad.example.org.

If neither the mail LDAP attribute is set nor a Mail application is bound to the user domain, no notification is sent.

To check for expiring passwords and immediately send notifications, run the following command on the leader node:

systemctl start password-warning.service

Default templates

Default templates are stored inside /etc/nethserver/password_warning directory. Available templates:

  • default_en.tmpl: English mail template
  • default_it.tmpl: Italian mail template
  • default_en.sbj: English mail subject
  • default_it.sbj: Italian mail subject

Configure password warning

The configuration can be done using the cluster web interface. If you want to configure it from command line, see the cluster/set-password-warning API.

The API takes the following parameters:

  • domain: the domain name
  • notification: enable or disable the notification
  • days: the number of days before the password expiration
  • mail_from: the email address of the sender
  • mail_template_name: the name of the template to use, it could be default_en, default_it or custom. If set to custom, you must provide also mail_template_content and mail_subject.
  • mail_template_content: the content of the template, must be base64 encoded
  • mail_subject: the subject of the email in plain text.

Using a Custom Template

Field mail_template_content and mail_subject use Python string template. The string representation can contain the following placeholders: $user, $name, $domain, $days, $portal_url.

  1. Create a file named custom.txt with your custom content: 
     Dear $user ($name) of domain $domain.
 Your password is going to expire in $days days.
 Change it here: $portal_url
  2. Use the custom template: 
     api-cli run cluster/set-password-warning --data '{"domain": "leader.cluster0.gs.nethserver.net", "notification": true, "days": 1000, "mail_from": "no-reply@nethserver.org", "mail_template_name": "custom", "mail_template_content": "'$(cat custom.txt | base64 -w0)'", "mail_subject": "my expiration"}'