search

Security

hashtag
GBDS Security Overview

GBDS has a security module that uses JWT tokens to perform user authentication and authorization. GBDS can be integrated with any LDAP-compatible user directory for authentication, and with Apache Ranger to provide authorization and access auditing.

hashtag
Components

There are four main components that work together to provide security for GBDS operations: API Security Module, Ranger, Solr and the LDAP Credentials Store.

  • API Security Module

    All API requests first pass through the security module, which enforces authentication and authorization for protected resources.

  • LDAP Credentials Store

    An LDAP instance (such as Active Directory, OpenLDAP) maintains information about users and user groups, and provides authentication to access the GBDS API and authorize API transactions.

  • Ranger

    Apache Ranger is a security framework that provides fine-grained access control to the GBDS API. Security policies can be created to block or allow access to endpoints of the API based on criteria such as user group, username, requested resources, HTTP methods used and IP address.

  • Solr

    GBDS uses Solr to store access logs for all API requests; through the Ranger control panel these logs can be viewed, searched and exported.

hashtag
Security-Enabled Workflows

hashtag
Authentication

There are two methods to authenticate with the API. When the client does not have a valid token, credential-based authentication must be used. In this case, the client makes a request to the token creation endpoint, passing a payload JSON with the valid credentials for a user registered in LDAP. Then, the Security Module creates a binding to the LDAP directory, using the bind user credentials (Bind User, see below), and performs authentication. It then creates an Authenticated Subject: a user associated with all the groups to which they belong. Finally, the Security Module builds a JWT token, digitally signs it and returns it to the client. This token can be used to make subsequent requests to the GBDS API.

Authentication Flow

When the client already has a valid token but it is near expiration, it is possible to use that token to create a new one with a later expiration time. First the client makes a request to the token creation endpoint with the valid token. The Security Module will validate the token and create a new one, using the same Authenticated Subject but with a new expirationTime. The new token will be sent to the user. Since this flow does not require querying the LDAP directory, it is much faster. Client applications should be designed to use token renewal whenever possible.

Authentication Flow

hashtag
Making an API Request

When a valid JWT token is acquired, it is sent to the GBDS API with every request, following the JWT standard. The request will have an authorization header with the Bearer, that is, Authorization: Bearer <token>. For protected resources, the Security Module will extract the token from the header and validate its signature using the original Signing Key. If the token is valid, the request is authenticated. Next, the Security Module will build an AuthorizationContext with the subject name, subject groups, request resources, IP address and HTTP method. This AuthorizationContext will be sent to Ranger to authorize or deny the request and will be logged by Solr, along with its results: Allowed or Denied. Finally, if the request is authorized, it will be forwarded to the GBDS API for processing.

Making an API Request

hashtag
Logout.

hashtag
Files

The security module uses four different configuration files:

  • /etc/griaule/conf/gbscluster.properties: Primary GBDS configuration file. When security is enabled, security options are configured in this file.

  • /etc/griaule/conf/gbsapi/keystore.jks: Keystore password-protected containing the Ldap Bind User Password and JWT Signing Key. Accessible only by the griaule user.

  • /etc/griaule/conf/gbsapi/truststore: Truststore password-protected containing the Ldap Server Certificate to allow LDAP connections. Accessible only by the griaule user.

  • /etc/griaule/conf/gbsapi/.password: Password file containing the passwords for the keystore and truststore. Passwords are stored in plain text, and it should only be accessible by the griaule user.

hashtag
Security Options

GBDS can run with or without security enabled, and this is controlled by the boolean configuration key: gbscluster.api.security.enabled. When this key is set to false, all security is disabled and all other security keys are ignored. If set to true, the following configuration keys MUST must also be configured:

General

  • gbscluster.api.security.keystore: path to the keystore generated from the generation script of the keystore

  • gbscluster.api.security.truststore: path to the truststore which contains the LDAP server certificate. Required to use LDAPS.

  • gbscluster.api.security.passwords: path to the file containing the passwords for the truststore and keystore. This file should have restricted access to the griaule user.

Authentication

  • gbscluster.api.security.ldap.url: URL to connect to the LDAP user store server.

  • gbscluster.api.security.ldap.userSearchBase: Distinguished Name (DN) of the user repository on the LDAP server. Example: ou=Users,dc=pd,dc=griaule

  • gbscluster.api.security.ldap.userSearchAttribute: User attribute that will be used as the username during authentication. Example: sAMAccountName, uid

  • gbscluster.api.security.ldap.userGroupMembershipAttribute: User attribute containing group membership information. Example: memberOf

  • gbscluster.api.security.ldap.bindUserDN Distinguished name of the bind user (Bind User). The bind user is a user account used by GBDS to bind to the LDAP server, authenticate users and fetch their groups.

Token

  • gbscluster.api.security.token.ttlInMilliseconds: Lifetime, in milliseconds, of the JWT token. Valid values: between 30 minutes and 3 hours.

Authorization

  • gbscluster.api.security.authorization.ranger.configurationDirectory: Directory with GBDS-specific ranger settings.

hashtag
Integrating with Active Directory

hashtag
Checking DNS Resolution

Before starting, make sure name resolution for the domain in which the cluster machines will be integrated is working correctly. For the domain named pd.griaule, use the following command:

circle-info

Make sure the ANSWER SECTION is present and contains the hostname and IP address of the domain controller.

circle-exclamation

Resolve any DNS resolution issues before continuing.

hashtag
Installing Required Packages

hashtag
Integrating with AD

Using the realm utility is the most convenient way to integrate SSSD with AD. Run the following command to show basic information about the domain:

hashtag
Join the Domain

To join, you will need an account that belongs to the domain administrators group.

Then, enable NSS and PAM with:

And, finally, verify that the machine successfully joined the domain by querying a user in the domain:

hashtag
Troubleshooting Errors

If any problem arises, use the following tips:

  • Recheck DNS name resolution. This is the most frequent cause of errors.

  • Make sure the directories /etc/sssd/ and /etc/sssd/conf.d exist and both have permission 0600.

  • Increase the debug_level in /etc/sssd/sssd.conf in all sessions, that is, [sssd], [nss], [pam], [domain]. Then, restart the sssd service: sudo service sssd restart. Logs will be generated in the folder /var/log/sssd/

  • In case of the error Insufficient permission to join the domain ...: the user used to join the domain does not have permission. Check that the user is part of the Domain Admins group

hashtag
Integrating with OpenLdap

This guide explains how to perform a minimal installation of OpenLDAP that will be compatible with GBDS for authentication and authorization.

hashtag
Install OpenLdap

hashtag
Dependencies

Connect to the LDAP host and install the required packages:

hashtag
Start Ldap Daemon

Next, start the LDAP Daemon and configure it to start automatically at system boot:

hashtag
Generate the root password

Run the command ldappasswd to create an LDAP root password. Write down both the plain password and the encoded version returned by ldappasswd.

hashtag
Create the hdb-database

The hdb is a hierarchical variant of the bdb database backend. In the configuration template below, set the following variables according to your use case:

  • olcSuffix: Represents the domain name for which the LDAP will provide account information. Additionally, this distinguished name will be added to the queries that will be sent to the backend.

  • olcRootDN: is the distinguished name of the user who will have administrator access to the LDAP server.

  • olcRootPW: Sets the administrator user's password. The password will be the encoded result of the command slappasswd, which was executed previously.

Put the configuration in a file called db.ldif.

circle-exclamation

MAKE SURE TO REPLACE olcRootPW, olcRootDN and olcSuffix.

circle-exclamation

Make sure there are no unnecessary spaces or blank lines, or problems may occur. See: https://serverfault.com/questions/578710/wrong-attributetype-when-using-ldapaddarrow-up-right

Send the changes to the server

circle-exclamation

If you do not use sudo, you will have permission problems: ldap_modify: Insufficient access (50)

hashtag
Restrict Monitoring Access to ldapadm

Put the following settings in a file called monitor.ldif and update the dn.base, where it says UPDATE ME, with the distinguished name of the value previously set for oclRootDN.

Send the changes to the server

circle-exclamation

If you do not use sudo, you will have permission problems: ldap_modify: Insufficient access (50)

hashtag
Add the hdb-backend configuration file

This file defines the basic configuration options for the hdb-backend:

hashtag
Add essential LDAP schemas

hashtag
Enable the memberOf module

circle-exclamation

THIS MUST BE DONE BEFORE ANY GROUP IS CREATED. THE MEMBEROF ATTRIBUTE IS AN OPERATIONAL ATTRIBUTE AND IS NOT CREATED RETROACTIVELY.

This module is required to query a user's groups. Put the following configuration in a file called memberof_config.ldif:

To send the changes to the server, run the command:

hashtag
Generate the database for your domain

Finally, we specify the database, which will use the previously created settings to serve all requests to the domain (value of olcSuffix). Update the DN of Users and Groups to match the domain settings. Put the settings in db_structure.ldif.

To send the changes to the server, use the configured administrator account (olcRootDN) after the -D. The ldapadd utility will ask for the password of the previously defined administrator account.

hashtag
Troubleshooting Errors

Note that error messages returned by OpenLdap can be confusing. If you have problems with the installation, it is recommended to enable debug mode. Stop the slapd service and then restart the process directly with the debug flag -d -1:

If TLS is enabled, also include the string for the ldaps endpoint:

hashtag
Encrypting OpenLdap passwords

By default, OpenLdap stores passwords in plain text. To change this behavior, it is necessary to: Set a global hash algorithm for passwords and enable the password policy to automatically encrypt any plain text password.

hashtag
Configuring the Hash Algorithm

We will use the SHA-512 algorithm with 50000 rounds and a 16-byte salt. Create a file called hash_algorithm.ldif

To send the settings to the server:

hashtag
Import the Password Policy Schema

hashtag
Load the Password Policy module

Create a file called password_policy_config.ldif containing:

Then send the changes with:

If you encounter an error indicating that pwdAttribute does not exist, this means the password policy schema was not imported.

hashtag
Creating a Default Password Policy

Create a file called password_policy.ldif containing:

Then send the changes with the command below, replacing the olcRootDN:

hashtag
Improving Security

hashtag
Disabling Anonymous Binding

Create a file disable_anon_binding.ldif containing:

Then send the changes:

To test if the changes worked, run the following anonymous search:

hashtag
Disabling Read Permission for User Passwords

Finally, any user can obtain another user’s password. Even if passwords are encrypted, this should be avoided. Especially if it is decided to keep passwords in plain text. So, create a file called disabled_password_read.ldif.

circle-exclamation

MAKE SURE THERE IS NO LINE BREAK FOR olcAccess

Send the changes:

If you run a bound search with any user that is not the olcRootDN, the only password returned will be that user’s own password.

hashtag
Enabling LDAPS

Before continuing, make sure your OpenLdap instance is working and that it has two organizational units: one for users and one for groups. You can do this using Apache Directory Studio to connect to the server using the ldap connection URL, for example: ldap://192.168.0.100:389arrow-up-right, and with your configured administrator account (olcRootDN).

To enable LDAPS, a valid certificate signed by a trusted authority is required. Alternatively, we provide a guide on how to set up your own certificate authority and how to use it to sign your server certificates.

hashtag
Create your CA

If you already have a valid certificate, you can skip this step.

On CentOS, you will find the file openssl.cnf, which contains the main configuration for your CA, in the directory /etc/pki/tls. However, our CA will be placed in /etc/pki/CA.

First, create the file index.txt and the serial files, which will contain information about signed certificates and the certificate serial number to be used, respectively.

Next, add the following policies to /etc/pki/tls/openssl.cnf:

Now, change to the CA directory:

We will generate the CA certificate, valid for 10 years, with the following command. The utility will ask you to set the private key password and information about the certificate subject (the CA in this case). Note this password, it will be required to sign certificates.

Then, restrict access rights to the CA private key:

hashtag
Create an LDAP Server Certificate

First, create a certificate signing request for the LDAP server:

Then, sign the server.csr using the CA private key:

Now, the ldap_cert.pem is the certificate for your server and your private key is in the file ldap_key.pem.

circle-info

the private key for the server is not encrypted (-nodes parameter). This is less secure, but OpenLdap does not support encrypted keys. So, it is essential to restrict all access to this file, as will be done in the following steps.

hashtag
Configuring OpenLdap to use the certificate and key

First, copy the CA certificate to /etc/openldap/cacerts. This will make OpenLdap trust the certificate signed by your CA.

Next, install your ldap server certificate in /etc/openldap/certs:

Setting permission 400 for the server private key is MANDATORY.

Create a file called enable_ldaps.ldif with the following settings:

Then, send the changes to the server and test the configuration files:

hashtag
Enable the TLS Endpoint

In the file /etc/sysconfig/slapdadd ldaps:/// to the line that starts with SLAPD_URLS=. After that, the line should be similar to:

hashtag
Force TLS

Create a file called force_tls.ldif with the following content:

Then, submit the changes to the server:

Now test the TLS requirement by running the following search. It should be denied due to the TLS requirement:

circle-exclamation

FROM NOW ON, ANY CALL TO ldapi:/// THAT REQUIRES olcRootDN (i.e. those that create entries in the directory tree) MUST BE MADE ON ldaps:///. CALLS THAT CAN USE -Y EXTERNAL (i.e., that modify cn=config) CAN STILL USE ldapi:///.

hashtag
Client-side TLS Configuration

To allow utilities such as ldapsearch to connect via TLS, add TLS_REQCERT allow address to /etc/openldap/ldap.conf:

hashtag
Create a Bind User for GBDS Ranger

GBDS and Ranger will require the user to bind to the LDAP server to perform queries and authentications. The only privilege this user needs is to read the directory tree.

circle-exclamation

Do not use olcRootDN as the bind user.

Create a file called bind_user.ldif with the following content. Replace the dn: for one that matches your installation. Additionally, set the bind user's password. If you enable password encryption, write the password in clear text. Otherwise, use slappasswd to produce a hash for the password and write the hash in the .ldif (Similar to how the password for olcRootDN).

Submit the changes with:

To test if the user was created correctly, run the following search authenticating with the bind user credentials:his

hashtag
LDAP API

The LDAP API is an API provided by Griaule to assist with user and group management. You can authenticate, create, delete or modify user data and list user and group data through the API.

circle-info

If the API files were not provided, contact the Griaule Support Team.

The full API can be accessed at LDAP API.

circle-exclamation

The API works with SSL security and requires a truststore and keystore. For more information on creating these, see the section Configuring GBDS Security.

hashtag
Configuring the API

To properly install and configure the API, follow the steps:

  1. Extract the provided package.

  2. Under /etc/griaule create the directory ldap.

  3. Move the directory conf from the provided package to the directory /etc/griaule/ldap.

  4. Move the config.properties to the directory /etc/griaule/ldap.

  5. Open the file config.properties and adjust the configuration parameters according to your environment.

  6. Under /var/lib/griaule create the directory ldap.

  7. Move the application bs-ldap-api.jar and the script kill-api.sh and start-api.sh to the new directory.

  8. Run the start script.

hashtag
Creating New Users and Groups

To create a new user, call Create Userarrow-up-right. The API will create the user, the password and the groups to which the user belongs.

If modifications to an existing user are required, call Modify User Dataarrow-up-right.

To retrieve user data, call Get User Dataarrow-up-right.

To add, modify or delete groups, access /etc/griaule/ldap/conf/group-list.json and modify the file as desired. You can also view all groups by calling List Groupsarrow-up-right.

hashtag
Installing Ranger

After OpenLDAP or Active Directory are configured, you can proceed to install Ranger.

hashtag
Prerequisites

hashtag
Installing ambari-infra

Go to Ambari and install the Ambari-infra service. This will provide a Solr instance to store audited data for Ranger.

hashtag
Install and Configure the MySQL Instance

  1. Install a MySQL instance

  2. Create the rangerdba user and grant the necessary privileges

    Note that Rangerdba$1 is the password for the rangerdba user. Replace it with the desired password.

  3. Test if the rangerdba user was created successfully: mysql -u rangerdba -p

  4. Install the MySQL connector: yum install mysql-connector-java*

  5. Register the connector with Ambari by running the following commands on the host with Ambari server installed: ambari-server setup --jdbc-db=mysql --jdbc-driver=/usr/share/java/mysql-connector-java.jar

hashtag
Installing the Ranger Service

Go to Ambari and add the Ranger service. Visit each of the following configuration tabs before finalizing the installation.

hashtag
Ranger Admin

  1. Leave the Ranger DB username as rangeradmin. Put whatever password you want, but save it for later.

  2. Make sure the options Setup Database and Database User are checked as Yes

  3. For the DBA username, use the user account created in the previous steps.

hashtag
Ranger User Information

This service will import users and groups into the Ranger database.

circle-info

Below, we give SUGGESTIONS for values in several required configuration options. You should ensure they match your LDAP deployment.

  1. Change the Sync Source to the Ldap/AD

  2. On the Common Config:

    1. LDAP/AD URL: If the URL configured here is for LDAPS, the LDAPS certificate will need to be imported into the usersync truststore as described in the section Importing the Ldaps Certificate

    2. Bind User: Fill in the bind user DN and password. The user only needs read privilege for the Users and Groups repository.

  3. On the User Configs:

    1. Username Attribute: attributes that store the userName for login. For AD, usually sAMAccountName and for OpenLDAP, uid.

    2. User Search Base: DN for the node that contains the user that needs to be synchronized (AD User).

  4. On the Group Configs:

    1. Group member Attribute: Group entity attribute that stores which user belongs to which group, in addition to information about the user-group membership. It is usually member

    2. Group Name Attribute: Group attribute that contains the group's name. Usually cn

    3. Group Object Class: Group entity class. For AD, it is usually group, for OpenLDAP it is usually groupOfNames

    4. Group Search Base: with the node that contains the groups that need to be synchronized

hashtag
Ranger Audit

  1. Check the option Audit to Solr

  2. Check the option Solr Cloud

hashtag
Installation Check

hashtag
User Synchronization

Go to the Ranger website at http://<host>:6080, log in and go to Settings, and then, Users/Groups. All users and groups from the User/Group Search Base should be present. If not, a user sync problem occurred. Check the log at /var/log/ranger/usersync/usersync.log to identify the error.

hashtag
Solr Audit

Go to the ambari-infra UI and check for the existence of the collection ranger_audits

hashtag
Enabling LDAP Integration with TLS

To enable Ranger to use LDAPS instead of LDAP, two changes need to be made. First, we need to update the LDAP connection URL to the TLS-protected URL. Second, we need to import the LDAPS certificate.

hashtag
Changing the LDAP Connection URL

Go, in Ambari, to Ranger, Configs, Ranger User Info, Common Configs. Change the LDAP/AD URL to ldaps://<hostname>:636

hashtag
Importing the Ldaps Certificate

If your LDAP repository is using TLS, you need to import its certificate into the UserSync truststore of ranger.

To obtain the location of the truststore, go to Ambari, Ranger, Configs, Advanced, Advanced ranger-ugsync-site, ranger.usersync.truststore.file. Then, use Java keytool to import the Ldaps certificate into the truststore. If the truststore already exists, a password will be required. If not, note the password used. You will need it later.

Next, fix the permission and owner of the truststore.

If the truststore is new, go to Ambari, Ranger, Configs, Advanced, Advanced ranger-ugsync-site and set the properties:

  1. ranger.usersync.truststore.file: with the absolute path to the truststore

  2. ranger.usersync.truststore.password: password for the truststore

hashtag
Troubleshooting Errors

If it is not possible to create the SSLContext for communication with the policy manager, there is a problem with the usersync credentials and keys. The only way to solve this problem is to delete the truststore /usr/hdp/current/ranger-usersync/conf/mytrustore.jks, recreate it with the Ldaps certificate and update the password configuration in Ambari, Ranger, Configs, Advanced ranger-usersync-site

hashtag
Configuring GBDS Security

Once Ranger, Solr and the LDAP server are installed and configured, we can proceed to GBDS configurations.

hashtag
Creating the GBDS Keystore

The keystore will be used to store bind user passwords and signing keys for the JWT tokens generated by the API.

Run the script /var/lib/griaule/gbsapi/scripts/create_keystore.py passing the path to the new keystore and the path to create the password file. The script will ask for the bind user's password to be saved. Then, the script will create the keystore with the bind user's password and a random signing key. The keystore will be protected by a randomly generated password, which will be saved in the specified password file. Finally, the keystore and password file will have their permissions changed to read-only.

circle-info

The password file contains passwords in plain text. Therefore, it is essential that this be kept in a secure location with access allowed only to the user griaule.

hashtag
Creating trustore for LDAPS

If you intend to connect to your LDAP server using SSL, you need to import the server's certificate into a truststore and configure GBDS to use it.

Assuming the server certificate is named server_cert.pem, run the following command and use a strong password for the truststore:

Besides creating the truststore, this command will restrict access to the griaule user. Then, save the truststore password of the

hashtag
sudo -u griaule chmod u+w /etc/griaule/conf/gbsapi/.passwords && sudo -u griaule echo "truststore=<YOUR_PASSWORD>" >> /etc/griaule/conf/gbsapi/.passwords && sudo -u griaule chmod 400 /etc/griaule/conf/gbsapi/.passwords

Edit the file /etc/griaule/conf/gbscluster.properties Updating the Configuration File

  1. gbscluster.api.security.enabledand modify the following properties: true : set to

  2. gbscluster.api.security.keystoreto enable security./etc/griaule/conf/gbsapi/keystore.jks)

  3. gbscluster.api.security.truststore: path to the keystore you created ( truststore : path to the/etc/griaule/conf/gbsapi/truststore)

  4. gbscluster.api.security.passwordsyou created (: path to the password file ()

  5. gbscluster.api.security.ldap.urletc/griaule/conf/gbsapi/.passwords : LDAP server URL, example:ldap://<host>:389 . If you enabled Ldaps, this should be

  6. gbscluster.api.security.ldap.userSearchBaseldaps://<host>:636 : The DN of the node in the tree that holds the users who will be enabled for authentication in GBDS. In your OpenLDAP installation guide, the DN should be:

    ou=Users,dc=oldap,dc=pd,dc=griaule

  7. gbscluster.api.security.ldap.userSearchAttributeIT MUST BE THE SAME DN CONFIGURED IN RANGER : User entity attribute that contains theuserName

    , which will be used during authentication.

  8. gbscluster.api.security.ldap.userGroupMembershipAttributeIT MUST BE THE SAME AS CONFIGURED IN RANGER

    , which will be used during authentication.

  9. gbscluster.api.security.ldap.bindUserDN: User entity attribute that contains the DNs of the groups it belongs to. Usually memberOf.

  10. gbscluster.api.security.token.ttlInMilliseconds: Bind user DN. Use the same one configured for Ranger.

hashtag
: Lifetime, in milliseconds, for the JWT token generated by the API. Valid values are between 30 minutes and 3 hours. We recommend 1 hour.

Installing the Ranger Authorization Service

To install and run the Ranger authorization service, you must have created a GBDS user and an ADMIN user. To create the administrator user, open the file in the folder administrator_group.ldif /etc/openldap

member: cn=gbds_bind,ou=Users,dc=oldap,dc=pd,dc=griaule Then, in the filegbds_user.ldif

cn: gbds_user

circle-info

sudo ldapadd -D "cn=ldapadm,dc=oldap,dc=pd,dc=griaule" -W -H ldapi:/// -f /etc/openldap/gbds_user.ldif ldapi:/// If TLS is enabled, remember to change ldaps:///.

to

java -jar /root/ranger-authorization-service-installer-1.0-SNAPSHOT-jar-with-dependencies.jar -admin_password 'Griaule.123' -admin_username admin -ranger_url http://127.0.0.1:6080 To finalize the process, update the Ranger XML Files. Go to /etc/griaule/conf/gbsapi/ and in the fileranger-gbds-audit.xml

<value>http://localhost:8886/solr/ranger_audits</value> Do the same for

hashtag
<value>http://localhost:6080</value>

Configuring PHP LDAP Admin

PHP LDAP Admin is a tool to manage OPENLDAP via GUI; it is an alternative to using the CLI. The user can add/modify/delete users, groups, roles, etc. via the GUI. Note that this is not required for LDAP to function.

Under First, install PHPLDAPAdmin dependencies./etc/httpd/conf.d then update the file with

Require all granted And also update the file in the folder config.php with

servers->setValue('login','attr','dn');

$ systemctl enable httpd.service && systemctl start httpd.service && systemctl status httpd.service

firewall-cmd --reload

setsebool -P httpd_can_connect_ldap on

Last updated

Was this helpful?