Apache Ranger™ and Ranger KMS

This manual is an installation guide for Apache Ranger™ and Ranger KMS.

This procedure applies to the GHDP environment.

Prerequisites

Install the prerequisites for the Ranger build.

Maven

Download the latest version of Maven at Downloading Apache Maven or:

cd /usr/local
wget https://dlcdn.apache.org/maven/maven-3/3.8.6/binaries/apache-maven-3.8.6-bin.tar.gz
tar -xvf apache-maven-<Version>-bin.tar.gz

Edit the file that loads the GHDP environment variables:

vim /etc/profile.d/hadoop_setup.sh

...
# MAVEN (to Ranger)
export M2_VERSION=$(ls -A /usr/local/ | grep apache-maven- | grep -v .gz | awk -F '-' '{print $3}')
export M2_HOME=/usr/local/apache-maven-$M2_VERSION
export M2=$M2_HOME/bin
...

Verify that the installation was successful:
```
mvn -version
```

Other requirements

Install the other required items:

yum -y install git
yum -y install gcc
yum -y install g++
yum install bzip2 -y
yum -y install java-1.8.0-openjdk-devel
yum -y install python3

Ranger Build

Download the source of the most up-to-date Ranger that suits your OS and Java version, from the official Ranger site or:
```
wget https://dlcdn.apache.org/ranger/2.3.0/apache-ranger-2.3.0.tar.gz
tar -xvf apache-ranger-2.3.0.tar.gz
cd ./apache-ranger-2.3.0
```
Run the build of Ranger using Maven:
```
mvn clean compile package install
```

If an insecure access error occurs due to an expired certificate on some repository link, run the build as follows:

mvn clean compile package install -Dmaven.wagon.http.ssl.insecure=true -Dmaven.wagon.http.ssl.allowall=true -Dmaven.wagon.http.ssl.ignore.validity.dates=true

Finish the build procedure with the following command:
```
mvn eclipse:eclipse
```

At the end, a folder called target with all Ranger components will be generated.

cd ./target
ls -l

total 1328820
drwxr-xr-x. 2 root root      4096 Dec 15 14:34 antrun
-rw-r--r--. 1 root root        87 Dec 15 14:34 checkstyle-cachefile
-rw-r--r--. 1 root root      9216 Dec 15 14:34 checkstyle-checker.xml
-rw-r--r--. 1 root root     20369 Dec 15 14:34 checkstyle-header.txt
-rw-r--r--. 1 root root        81 Dec 15 14:34 checkstyle-result.xml
-rw-r--r--. 1 root root      1144 Dec 15 14:34 checkstyle-suppressions.xml
drwxr-xr-x. 3 root root      4096 Dec 15 14:34 maven-shared-archive-resources
-rw-r--r--. 1 root root 518758611 Dec 15 14:34 ranger-2.3.0-admin.tar.gz
-rw-r--r--. 1 root root  41566842 Dec 15 14:34 ranger-2.3.0-atlas-plugin.tar.gz
-rw-r--r--. 1 root root  36041635 Dec 15 14:34 ranger-2.3.0-elasticsearch-plugin.tar.gz
-rw-r--r--. 1 root root  36975553 Dec 15 14:34 ranger-2.3.0-hbase-plugin.tar.gz
-rw-r--r--. 1 root root  35537921 Dec 15 14:34 ranger-2.3.0-hdfs-plugin.tar.gz
-rw-r--r--. 1 root root  35327622 Dec 15 14:34 ranger-2.3.0-hive-plugin.tar.gz
-rw-r--r--. 1 root root  54580246 Dec 15 14:34 ranger-2.3.0-kafka-plugin.tar.gz
drwxr-xr-x. 7 root root      4096 Dec 15 14:34 ranger-2.3.0-kms
-rw-r--r--. 1 root root 195191513 Dec 15 14:34 ranger-2.3.0-kms.tar.gz
-rw-r--r--. 1 root root  49243221 Dec 15 14:34 ranger-2.3.0-knox-plugin.tar.gz
-rw-r--r--. 1 root root  34477047 Dec 15 14:34 ranger-2.3.0-kylin-plugin.tar.gz
-rw-r--r--. 1 root root     34007 Dec 15 14:34 ranger-2.3.0-migration-util.tar.gz
-rw-r--r--. 1 root root  41233187 Dec 15 14:34 ranger-2.3.0-ozone-plugin.tar.gz
-rw-r--r--. 1 root root  55205632 Dec 15 14:34 ranger-2.3.0-presto-plugin.tar.gz
-rw-r--r--. 1 root root  15803444 Dec 15 14:34 ranger-2.3.0-ranger-tools.tar.gz
-rw-r--r--. 1 root root    905882 Dec 15 14:34 ranger-2.3.0-schema-registry-plugin.jar
-rw-r--r--. 1 root root     37302 Dec 15 14:34 ranger-2.3.0-solr_audit_conf.tar.gz
-rw-r--r--. 1 root root     40595 Dec 15 14:34 ranger-2.3.0-solr_audit_conf.zip
-rw-r--r--. 1 root root  36130187 Dec 15 14:34 ranger-2.3.0-solr-plugin.tar.gz
-rw-r--r--. 1 root root  34715214 Dec 15 14:34 ranger-2.3.0-sqoop-plugin.tar.gz
-rw-r--r--. 1 root root   6315989 Dec 15 14:34 ranger-2.3.0-src.tar.gz
-rw-r--r--. 1 root root  49575156 Dec 15 14:34 ranger-2.3.0-storm-plugin.tar.gz
-rw-r--r--. 1 root root  30112906 Dec 15 14:34 ranger-2.3.0-tagsync.tar.gz
-rw-r--r--. 1 root root  19205167 Dec 15 14:34 ranger-2.3.0-usersync.tar.gz
-rw-r--r--. 1 root root  33381584 Dec 15 14:34 ranger-2.3.0-yarn-plugin.tar.gz
-rw-r--r--. 1 root root    196038 Dec 15 14:34 rat.txt
-rw-r--r--. 1 root root         5 Dec 15 14:34 version

Solr Installation

Refer to the Official Solr Site for the best Solr version for your system.

Go to the build Ranger previous topic.
Inside that folder, go to the Solr installer folder, where it will be pre-configured for Ranger use:
```
cd ~/apache-ranger-2.3.0
cd ./security-admin/contrib/solr_for_audit_setup/
```
Create the Solr folder according to the chosen version:
```
mkdir -p /usr/gdp/hadoop/solr/8.11.2/
```

Edit the file install.properties:

vim install.properties

...
SOLR_INSTALL=true
SOLR_DOWNLOAD_URL=https://dlcdn.apache.org/lucene/solr/8.11.2/solr-8.11.2.tgz
SOLR_LOG_FOLDER=/var/log/hadoop/solr/ranger_audits
...

:wq

sed -i 's/\/opt\/solr/\/usr\/gdp\/hadoop\/solr\/8.11.2/g' install.properties

Run the script setup.sh and check the start procedures as indicated by the installation log:

chmod +x setup.sh
./setup.sh

less /usr/gdp/hadoop/solr/8.11.2/ranger_audit_server/install_notes.txt

Start Solr:

/usr/gdp/hadoop/solr/8.11.2/ranger_audit_server/scripts/start_solr.sh

Ranger Admin installation and configuration

Create the Ranger Admin folder:

mkdir -p /usr/gdp/hadoop/ranger/2.3.0/ranger-admin

Go to the folder targetgenerated in the buildprocedure, and unzip the file ranger-2.3.0-admin.tar.gz:
```
cd ./apache-ranger-2.3.0/target
tar -xvf ranger-2.3.0-admin.tar.gz
```

Copy all files inside the unzipped folder to the ranger-admin.

cd ranger-2.3.0-admin
cp -R * /usr/gdp/hadoop/ranger/2.3.0/ranger-admin/

In the database, create the user rangerdba as follows:

mysql -uroot -p

SET GLOBAL validate_password_policy=LOW;

CREATE USER 'rangerdba'@'localhost' IDENTIFIED BY 'rangerdba';

GRANT ALL PRIVILEGES ON *.* TO 'rangerdba'@'localhost';

CREATE USER 'rangerdba'@'%' IDENTIFIED BY 'rangerdba';

GRANT ALL PRIVILEGES ON *.* TO 'rangerdba'@'%';

GRANT ALL PRIVILEGES ON *.* TO 'rangerdba'@'localhost' WITH GRANT OPTION;

GRANT ALL PRIVILEGES ON *.* TO 'rangerdba'@'%' WITH GRANT OPTION;

FLUSH PRIVILEGES;

If not installed, install the mysql-connector-java and verify that the file mysql-connector-java.jar is in the correct folder:
```
yum install mysql-connector-java
ls /usr/share/java/mysql-connector-java.jar
```

Create the logs folder for Ranger Admin:

mkdir -p /var/log/hadoop/ranger/ranger-admin

In the Ranger Admin folder, edit the file install.properties:

cd /usr/gdp/hadoop/ranger/2.3.0/ranger-admin/
vim install.properties

...
db_root_user=rangerdba
db_root_password=rangerdba
db_host=localhost

db_name=ranger
db_user=rangerdba
db_password=rangerdba

rangerAdmin_password=Griaule.123
rangerTagsync_password=Griaule.123
rangerUsersync_password=Griaule.123
keyadmin_password=Griaule.123

audit_solr_urls=http://localhost:6083/solr/ranger_audits

policymgr_supportedcomponents=hbase,hdfs,kafka,kms

authentication_method=UNIX
remoteLoginEnabled=true
authServiceHostName=localhost
authServicePort=5151

hadoop_conf=/etc/hadoop/hdfs/conf/

RANGER_ADMIN_LOG_DIR=/var/log/hadoop/ranger/ranger-admin
...

Run the setup:
```
./setup.sh
```

Add permissions for the Ranger folders and logs and add the user ranger to the group hadoop.

chown -R ranger: /usr/gdp/hadoop/ranger/
chown -R ranger: /var/log/hadoop/ranger/
usermod -a -G hadoop ranger

To start Ranger Admin use the command:
```
ranger-admin start
```
Access the link and enter the user admin and the pre-configured password.
```
http://<my_ip>:6080/
```

In this context, the pre-configured password will always be Griaule.123.

Ranger UserSync Installation

In the Ranger build folder, create a folder for Ranger UserSync called ranger-usersyncand unzip the tar.gz related to the application and copy all files to the created folder:
```
mkdir -p /usr/gdp/hadoop/ranger/2.3.0/ranger-usersync
tar -xvf ranger-2.3.0-usersync.tar.gz
cd ranger-2.3.0-usersync
cp -R * /usr/gdp/hadoop/ranger/2.3.0/ranger-usersync
```

Create the logs and grant the user ranger access to the folders /usr/gdp/hadoop/ranger/ and /var/log/hadoop/ranger/:

mkdir -p /var/log/hadoop/ranger/ranger-usersync
chown -R ranger: /usr/gdp/hadoop/ranger/
chown -R ranger: /var/log/hadoop/ranger/

In the ranger-usersyncfolder, edit the file install.properties as follows:

cd /usr/gdp/hadoop/ranger/2.3.0/ranger-usersync
vim install.properties

...
POLICY_MGR_URL = http://<ip_addr>:6080

SYNC_SOURCE = unix

SYNC_INTERVAL = 5

rangerUsersync_password=Griaule.123 # same password that was set for it in ranger-admin

hadoop_conf=/etc/hadoop/hdfs/conf

logdir=/var/log/hadoop/ranger/ranger-usersync
...

Change the path default of the application from /etc/ranger address to /usr/gdp/hadoop/ranger/2.3.0/ranger-usersync/ranger:
```
sed -i 's/\/etc\/ranger/\/usr\/gdp\/hadoop\/ranger\/2.3.0\/ranger-usersync\/ranger/g' install.properties
```
Run the script setup.sh:
```
./setup.sh
```

Change the configuration to enable UserSync synchronization:

vim /usr/gdp/hadoop/ranger/2.3.0/ranger-usersync/conf/ranger-ugsync-site.xml

<property>
  <name>ranger.usersync.enabled</name>
  <value>true</value>
</property>

After the installation with result successfully, start the service using the script ranger-usersync-services.sh:
```
# Start:
./ranger-usersync-services.sh start

# Stop:
./ranger-usersync-services.sh stop
```

Plugin Installation

The plugins are not required for the operation of Ranger KMS. They are only features available for auditing Hadoop resources.

HDFS Plugin

The HDFS Plugin must be installed on all the NameNodes.

Create the folder ranger-hdfs-plugin according to the GHDP structure:
```
mkdir -p /usr/gdp/hadoop/ranger/2.3.0/ranger-hdfs-plugin
```

In the Ranger build of Ranger, unzip the file ranger-2.3.0-hdfs-plugin.tar.gz and copy all files to the folder created previously:

cd ./apache-ranger-2.3.0/target/
tar -xvf ranger-2.3.0-hdfs-plugin.tar.gz
cd ranger-2.3.0-hdfs-plugin
cp -R * /usr/gdp/hadoop/ranger/2.3.0/ranger-hdfs-plugin

In the plugin folder, edit the file install.properties:

cd /usr/gdp/hadoop/ranger/2.3.0/ranger-hdfs-plugin
vim install.properties

...
POLICY_MGR_URL=http://<host_addrs>:6080

REPOSITORY_NAME=hadoopdev

COMPONENT_INSTALL_DIR_NAME=/usr/gdp/hadoop/hdfs/3.2.4/

XAAUDIT.SOLR.ENABLE = true
XAAUDIT.SOLR.URL = http://<host_addrs>:6083/solr/ranger_audits
XAAUDIT.SOLR.USER = NONE
XAAUDIT.SOLR.PASSWORD = NONE
XAAUDIT.SOLR.ZOOKEEPER = NONE
XAAUDIT.SOLR.FILE_SPOOL_DIR = /var/log/hadoop/hdfs/audit/solr/spool
...

If there is more than one NameNode, create the same folder structure and copy all content to the other NameNodes with scp:

This procedure must be performed before enabling the plugin.

# NameNode2
mkdir -p /usr/gdp/hadoop/ranger/2.3.0/ranger-hdfs-plugin

# NameNode1
scp -r * root@localhost:/usr/gdp/hadoop/ranger/2.3.0/ranger-hdfs-plugin/

# NameNode2
cd /usr/gdp/hadoop/ranger/2.3.0/ranger-hdfs-plugin
ls -l

Enable the plugin running the script enable-hdfs-plugin.sh:
```
./enable-hdfs-plugin.sh
```
Connect to the Ranger Admin UI. On the main screen, under HDFS, click the + button and fill the fields with the following information:
- Service Name: hadoopdev
- Display Name: hadoopdev
- Username: hadoop (UNIX User)
- Password: <password created for the hadoop user on UNIX>
- NameNode URL: hdfs://localhost:50070
- Authentication Type: Simple
Keep the rest of the settings unchanged and click the Add button.
Restart the cluster.

HBase Plugin

The HBase Plugin must be installed on all hosts with Master and Regional.

Create the folder ranger-hbase-plugin according to the GHDP structure.
```
mkdir -p /usr/gdp/hadoop/ranger/2.3.0/ranger-hbase-plugin
```

In the Ranger build of Ranger, unzip the file ranger-2.3.0-hbase-plugin.tar.gz and copy all files to the folder created previously:

cd ./apache-ranger-2.3.0/target/
tar -xvf ranger-2.3.0-hbase-plugin.tar.gz
cd ranger-2.3.0-hbase-plugin
cp -R * /usr/gdp/hadoop/ranger/2.3.0/ranger-hbase-plugin

In the plugin folder, edit the file install.properties:

vim install.properties

...
POLICY_MGR_URL=http://localhost:6080

REPOSITORY_NAME=hbasedev

COMPONENT_INSTALL_DIR_NAME=/usr/gdp/hadoop/hbase/2.5.1

XAAUDIT.SOLR.ENABLE=true
XAAUDIT.SOLR.URL=http://localhost:6083/solr/ranger_audits
XAAUDIT.SOLR.USER=NONE
XAAUDIT.SOLR.PASSWORD=NONE
XAAUDIT.SOLR.ZOOKEEPER=NONE
XAAUDIT.SOLR.FILE_SPOOL_DIR=/var/log/hadoop/hbase/audit/solr/spool

XAAUDIT.SOLR.IS_ENABLED=true
XAAUDIT.SOLR.MAX_QUEUE_SIZE=1
XAAUDIT.SOLR.MAX_FLUSH_INTERVAL_MS=1000
XAAUDIT.SOLR.SOLR_URL=http://localhost:6083/solr/ranger_audits
...

Create the same folder structure and copy all content to the Master and Regional:

This procedure must be performed before enabling the plugin.

# node2 & node3
mkdir -p /usr/gdp/hadoop/ranger/2.3.0/ranger-hbase-plugin

# node1
scp -r * root@localhost:/usr/gdp/hadoop/ranger/2.3.0/ranger-hbase-plugin/
scp -r * root@localhost:/usr/gdp/hadoop/ranger/2.3.0/ranger-hbase-plugin/

# node2 & node3
cd /usr/gdp/hadoop/ranger/2.3.0/ranger-hbase-plugin
ls -l

Create a user hbase and enable the plugin running the script enable-hbase-plugin.sh:
```
useradd hbase
passwd hbase
./enable-hbase-plugin.sh
```
Connect to the Ranger Admin UI. On the main screen, under HDFS, click the + button and fill the fields with the following information:
- Service Name: hadoopdev
- Display Name: hadoopdev
- Username: hbase (UNIX User)
- Password: <password created for the hbase user on UNIX>
- hadoop.security.authentication: Simple
- hbase.security.authentication: Simple
- hbase.zookeeper.property.clientPort: 2181
- hbase.zookeeper.quorum: ,,
- zookeeper.znode.parent: /hbase-unsecure
Keep the rest of the settings unchanged and click the Add button.
Restart the cluster.

Kafka Plugin

The Kafka Plugin must be installed on all hosts that have the component installed.

Create the folder ranger-kafka-plugin according to the GHDP structure:
```
mkdir -p /usr/gdp/hadoop/ranger/2.3.0/ranger-kafka-plugin
```

In the Ranger build of Ranger, unzip the file ranger-2.3.0-kafka-plugin.tar.gz and copy all files to the folder created previously:

cd ./apache-ranger-2.3.0/target/
tar -xvf ranger-2.3.0-kafka-plugin.tar.gz
cd ranger-2.3.0-kafka-plugin
cp -R * /usr/gdp/hadoop/ranger/2.3.0/ranger-kafka-plugin

In the plugin folder, edit the file install.properties:

vim install.properties

...
COMPONENT_INSTALL_DIR_NAME=/usr/gdp/hadoop/kafka/3.3.1/

POLICY_MGR_URL=http://localhost:6080

REPOSITORY_NAME=kafkadev

XAAUDIT.SOLR.ENABLE=true
XAAUDIT.SOLR.URL=http://localhost:6083/solr/ranger_audits
XAAUDIT.SOLR.USER=NONE
XAAUDIT.SOLR.PASSWORD=NONE
XAAUDIT.SOLR.ZOOKEEPER=NONE
XAAUDIT.SOLR.FILE_SPOOL_DIR=/var/log/hadoop/kafka/audit/solr/spool

XAAUDIT.SOLR.IS_ENABLED=true
XAAUDIT.SOLR.MAX_QUEUE_SIZE=1
XAAUDIT.SOLR.MAX_FLUSH_INTERVAL_MS=1000
XAAUDIT.SOLR.SOLR_URL=http://localhost:6083/solr/ranger_audits
...

Create the same folder structure and copy all content to the other nodes:

This procedure must be performed before enabling the plugin.

# node2 & node3
mkdir -p /usr/gdp/hadoop/ranger/2.3.0/ranger-kafka-plugin

# node1
scp -r * root@localhost:/usr/gdp/hadoop/ranger/2.3.0/ranger-kafka-plugin
scp -r * root@localhost:/usr/gdp/hadoop/ranger/2.3.0/ranger-kafka-plugin

# node2 & node3
cd /usr/gdp/hadoop/ranger/2.3.0/ranger-kafka-plugin
ls -l

Create a user kafka and enable the plugin running the script enable-kafka-plugin.sh:
```
useradd kafka
passwd kafka
./enable-kafka-plugin.sh
```
Connect to the Ranger Admin UI. On the main screen, under HDFS, click the + button and fill the fields with the following information:
- Service Name: hadoopdev
- Display Name: hadoopdev
- Username: hbase (UNIX User)
- Password: <password created for the hbase user on UNIX>
- hadoop.security.authentication: Simple
- hbase.security.authentication: Simple
- hbase.zookeeper.property.clientPort: 2181
- hbase.zookeeper.quorum: ,,
- zookeeper.znode.parent: /hbase-unsecure
Keep the rest of the settings unchanged and click the Add button.
Restart the cluster.

Ranger KMS Installation and Configuration

Ranger KMS Installation

Create the folder ranger-kms according to the GHDP structure:
```
mkdir -p /usr/gdp/hadoop/ranger/2.3.0/ranger-kms
```

On the MySQL server, create a user rangerkms for database management by the application:

mysql -uroot -p

CREATE USER 'rangerkms'@'localhost' IDENTIFIED BY 'rangerkms';

GRANT ALL PRIVILEGES ON *.* TO 'rangerkms'@'localhost';

CREATE USER 'rangerkms'@'%' IDENTIFIED BY 'rangerkms';

GRANT ALL PRIVILEGES ON *.* TO 'rangerkms'@'%';

GRANT ALL PRIVILEGES ON *.* TO 'rangerkms'@'localhost' WITH GRANT OPTION;

GRANT ALL PRIVILEGES ON *.* TO 'rangerkms'@'%' WITH GRANT OPTION;

FLUSH PRIVILEGES;

In the Ranger build of Ranger, unzip Ranger KMS and copy all files to the folder created previously:

cd ./apache-ranger-2.3.0/target/
tar -xvf ranger-2.3.0-kms.tar.gz
cd ranger-2.3.0-kms
cp -R * /usr/gdp/hadoop/ranger/2.3.0/ranger-kms/

Create the logs for Ranger KMS:

mkdir -p /var/log/hadoop/ranger/ranger-kms/

Using a password generator, create a password with the following parameters and store it in a safe place (it will be used in the next step):
- 16 characters
- Uppercase letters
- Lowercase letters
- Special characters.

In the Ranger KMS folder, edit the file install.properties adding configuration for Java Key Store (stores the master key in a file on the server itself):

Use the 16-character password generated in the previous step as KMS_MASTER_KEY_PASSWD. For example: $ZH1$Q8&ExUaTku8.

cd /usr/gdp/hadoop/ranger/2.3.0/ranger-kms/
vim install.properties

...
db_root_user=rangerkms
db_root_password=rangerkms
db_host=localhost

db_name=rangerkms
db_user=rangerkms
db_password=rangerkms

COMPONENT_INSTALL_DIR_NAME=/usr/gdp/hadoop/ranger/2.3.0/ranger-kms

KMS_MASTER_KEY_PASSWD=<16-character password generated previously>

hadoop_conf=/etc/hadoop/hdfs/conf

POLICY_MGR_URL=http://localhost:6080

REPOSITORY_NAME=kmsdev

XAAUDIT.SOLR.ENABLE=true
XAAUDIT.SOLR.URL=http://localhost:6083/solr/ranger_audits
XAAUDIT.SOLR.USER=NONE
XAAUDIT.SOLR.PASSWORD=NONE
XAAUDIT.SOLR.ZOOKEEPER=NONE
XAAUDIT.SOLR.FILE_SPOOL_DIR=/var/log/hadoop/ranger/ranger-kms/audit/solr/spool

XAAUDIT.SOLR.IS_ENABLED=true
XAAUDIT.SOLR.MAX_QUEUE_SIZE=1
XAAUDIT.SOLR.MAX_FLUSH_INTERVAL_MS=1000
XAAUDIT.SOLR.SOLR_URL=http://localhost:6083/solr/ranger_audits

RANGER_KMS_LOG_DIR=/var/log/hadoop/ranger/ranger-kms
...

Ranger KMS Configuration with Luna Cloud HSM

Before performing the setup of Ranger KMS, it is necessary to add the LunaProvider in the java.security. To do this, edit the file java.security, which is located in the folder <JDK_installation_directory>/jre/lib/security, adding two lines at the end: one with the LunaProvider in the provider list sequence, security.provider.10=com.safenetinc.luna.provider.LunaProvider, and one with the setting for Luna to work, com.safenetinc.luna.provider.createExtractableKeys=true:

vim /usr/lib/java/jre/lib/security/java.security

#
# List of providers and their preference orders (see above):
#
security.provider.1=sun.security.provider.Sun
security.provider.2=sun.security.rsa.SunRsaSign
security.provider.3=sun.security.ec.SunEC
security.provider.4=com.sun.net.ssl.internal.ssl.Provider
security.provider.5=com.sun.crypto.provider.SunJCE
security.provider.6=sun.security.jgss.SunProvider
security.provider.7=com.sun.security.sasl.Provider
security.provider.8=org.jcp.xml.dsig.internal.dom.XMLDSigRI
security.provider.9=sun.security.smartcardio.SunPCSC
security.provider.10=com.safenetinc.luna.provider.LunaProvider

com.safenetinc.luna.provider.createExtractableKeys=true

Copy the files LunaProvider.jar and libLunaAPI.so to the folder <JDK_installation_directory/jre/lib/ext.

cp /usr/safenet/lunaclient/jsp/LunaProvider.jar /usr/lib/java/jre/lib/ext/
cp /usr/safenet/lunaclient/jsp/64/libLunaAPI.so /usr/lib/java/jre/lib/ext/

Using a password generator, create a password with the following parameters and store it in a safe place (it will be used in the next step):
- 16 characters
- Uppercase letters
- Lowercase letters
- Special characters.

Edit the file install.properties to the setup of Ranger KMS with Luna Cloud HSM:

Use the 16-character password generated in the previous step as KMS_MASTER_KEY_PASSWD. For example: $ZH1$Q8&ExUaTku8.

cd /usr/gdp/hadoop/ranger/2.3.0/ranger-kms/
vim install.properties

...
db_root_user=rangerkms
db_root_password=rangerkms
db_host=localhost

db_name=rangerkms
db_user=rangerkms
db_password=rangerkms

COMPONENT_INSTALL_DIR_NAME=/usr/gdp/hadoop/ranger/2.3.0/ranger-kms

KMS_MASTER_KEY_PASSWD=<16-character password generated previously>

hadoop_conf=/etc/hadoop/hdfs/conf

#------------------------- Ranger KMS HSM CONFIG ------------------------------
HSM_TYPE=LunaProvider
HSM_ENABLED=true
HSM_PARTITION_NAME=rangerkms
HSM_PARTITION_PASSWORD=Griaule.123

POLICY_MGR_URL=http://localhost:6080

REPOSITORY_NAME=kmsdev

XAAUDIT.SOLR.ENABLE=true
XAAUDIT.SOLR.URL=http://localhost:6083/solr/ranger_audits
XAAUDIT.SOLR.USER=NONE
XAAUDIT.SOLR.PASSWORD=NONE
XAAUDIT.SOLR.ZOOKEEPER=NONE
XAAUDIT.SOLR.FILE_SPOOL_DIR=/var/log/hadoop/ranger/ranger-kms/audit/solr/spool

XAAUDIT.SOLR.IS_ENABLED=true
XAAUDIT.SOLR.MAX_QUEUE_SIZE=1
XAAUDIT.SOLR.MAX_FLUSH_INTERVAL_MS=1000
XAAUDIT.SOLR.SOLR_URL=http://localhost:6083/solr/ranger_audits

RANGER_KMS_LOG_DIR=/var/log/hadoop/ranger/ranger-kms
...

In all the nodes, so that the datanodes can access the KMS, edit the file core-site.xml changing the value of the property hadoop.security.key.provider.path address to kms://http@localhost:9292/kms:
```
vim /etc/hadoop/hdfs/conf/core-site.xml
```
```
<property>
  <name>hadoop.security.key.provider.path</name>
  <value>kms://http@localhost:9292/kms</value>
</property>
```
Restart HDFS.
```
dfsstop
dfsstart
```

Grant the user kms permissions for the folders:

chown -R kms: /var/log/hadoop/ranger/ranger-kms/
chown -R kms: /usr/gdp/hadoop/ranger/2.3.0/ranger-kms/

Run the script of setup, wait for the installation to finish with the message successfully and start Ranger KMS:
```
./setup.sh

# Start:
ranger-kms start

# Stop:
ranger-kms stop
```
If everything went successfully, it will be possible to access the Ranger KMS dashboard through the Ranger Admin address using the user keyadmin and the password defined in the Ranger Admin installation procedure.
- Link: http://<my_ip>:6080/
- User: keyadmin
- Password: <defined in install.properties during Ranger Admin setup>
Log into the Ranger Admin UI with the user admin, go to Settings > Users/Groups/Roles. On the Users tab, click the Add New User button and create the users:
- hive
- hdfs
- om
- hbase
Then, log out and log in as keyadmin to access the Ranger KMS UI dashboard in the KMS Service. Click the + button to create the repository kmsdev, according to the specifications below:
- Service Name: kmsdev
- KMS URL: kms://http@:9292/kms
- Username: keyadmin
- Password: <password set in the Ranger Admin installation procedure>
On the same screen, in Audit Filter, click the + button to add an ACL with the following specifications:
- Access Result: ALLOWED
- Permissions: Select All
- Users: keyadmin
Click Add. Then, click to edit the repository kmsdev and click the Test Connection button to confirm if the entire procedure ran correctly.
Restart Ranger KMS:
```
ranger-kms stop
ranger-kms start
```
If you are using Luna Cloud HSM, verify whether the master keywas created. To do this, run the lunacm:
```
lunacm
```
Or:
```
cd /usr/safenet/lunaclient/
./bin/64/lunacm
```
Log in with the role crypto officer:
```
role login -name crypto officer
```

List the partition contents to check if the master key was created successfully:

partition contents

Example output with the master key created:

lunacm:>partition contents

        The 'Crypto Officer' is currently logged in. Looking for objects
        accessible to the 'Crypto Officer'.

        Object list:

        Label:         RangerKMSKey
        Handle:        1
        Object Type:   Symmetric Key
        Usage Limit:   none
        Object UID:    9f1d00002d000001706c0800

        Number of objects:  1

Command Result : No Error

The Ranger KMS configuration file, dbks-site.xml, is located at:

/usr/gdp/hadoop/ranger/2.3.0/ranger-kms/ews/webapp/WEB-INF/classes/conf/dbks-site.xml

Enable Load Balancer in Ranger KMS

On a second server, perform the entire Ranger KMS installation.

If you are using Luna Cloud HSM, follow the instructions in the Luna Cloud HSM installation guide contained in the topics 1.1. Client download and 1.2. Client installation on the server to perform the client installation. It is not necessary to perform the procedures to activate slot, start partition, or start the roles contained in later topics of the guide. Then, strictly follow the instructions for installation and configuration of the HSM with Ranger, but taking the precautions described below.

On the step 5 of the configuration, in which it is necessary to change the file core-site.xml so that the datanodes can access the KMS, proceed as follows: open the file, find the property hadoop.security.key.provider.path and change its value of kms://http@localhost:9292/kms address to kms://http@host1;host2:9292/kms:
```
vim /etc/hadoop/hdfs/conf/core-site.xml
```
Before:
```
<property>
  <name>hadoop.security.key.provider.path</name>
  <value>kms://http@localhost:9292/kms</value>
</property>
```
After:
```
<property>
  <name>hadoop.security.key.provider.path</name>
  <value>kms://http@host1;host2:9292/kms</value>
</property>
```
Restart HDFS:
```
dfsstop
dfsstart
```
Continue with the completion of the Ranger KMS configuration, starting from step 6.

Ranger Uninstallation

To uninstall Ranger, run the following commands:

rm -rf /var/log/ranger /var/log/hadoop/ranger /usr/gdp/hadoop/solr/ /usr/gdp/hadoop/ranger/ /etc/ranger/ /var/log/hadoop/solr/ /var/lib/mysql/ranger/

rm -f /usr/gdp/hadoop/*/*/.ranger*
rm -f /usr/gdp/hadoop/*/*/*/.ranger*
rm -f /usr/gdp/hadoop/*/*/*/*/.ranger*

rm -f /usr/gdp/hadoop/hdfs/3.2.4/etc/hadoop/ranger*
rm -f /usr/gdp/hadoop/hdfs/3.2.4/share/hadoop/hdfs/lib/ranger*

rm -f /usr/bin/ranger*
rm -f /etc/rc.d/init.d/ranger*
rm -f /etc/rc.d/rc2.d/*ranger*
rm -f /etc/rc.d/rc3.d/*ranger*

mysql -uroot -p
show schemas;
drop database ranger;
drop database rangerkms;

Last updated 7 months ago

Was this helpful?

hashtagPrerequisites

hashtagMaven

hashtagOther requirements

hashtagRanger Build

hashtagSolr Installation

hashtagRanger Admin installation and configuration

hashtagRanger UserSync Installation

hashtagPlugin Installation

hashtagHDFS Plugin

hashtagHBase Plugin

hashtagKafka Plugin

hashtagRanger KMS Installation and Configuration

hashtagRanger KMS Installation

hashtagRanger KMS Configuration with Luna Cloud HSM

hashtagEnable Load Balancer in Ranger KMS

hashtagRanger Uninstallation

Prerequisites

Maven

Other requirements

Ranger Build

Solr Installation

Ranger Admin installation and configuration

Ranger UserSync Installation

Plugin Installation

HDFS Plugin

HBase Plugin

Kafka Plugin

Ranger KMS Installation and Configuration

Ranger KMS Installation

Ranger KMS Configuration with Luna Cloud HSM

Enable Load Balancer in Ranger KMS

Ranger Uninstallation