直接进入正题

1、下载相关安装包

Cloudera Manager 下载地址:

https://archive.cloudera.com/cm6/6.1.0/redhat7/yum/RPMS/x86_64/cloudera-manager-agent-6.1.0-769885.el7.x86_64.rpm

https://archive.cloudera.com/cm6/6.1.0/redhat7/yum/RPMS/x86_64/cloudera-manager-daemons-6.1.0-769885.el7.x86_64.rpm

https://archive.cloudera.com/cm6/6.1.0/redhat7/yum/RPMS/x86_64/cloudera-manager-server-6.1.0-769885.el7.x86_64.rpm

jdk 下载地址:
https://archive.cloudera.com/cm6/6.1.0/redhat7/yum/RPMS/x86_64/oracle-j2sdk1.8-1.8.0+update141-1.x86_64.rpm

CDH 安装包下载地址:https://archive.cloudera.com/cdh6/6.1.0/parcels/
需要下载以下文件:
CDH-6.1.0-1.cdh6.1.0.p0.770702-el7.parcel
manifest.json

2、安装三台服务器

IP地址主机名
192.168.1.211cdh1-master
192.168.1.212cdh2-slave
192.168.1.213cdh3-slave

3、修改主机名及映射主机名(所有节点/重启)

修改主机名:hostnamectl set-hostname
设置映射主机名,将以下内容添加到 /etc/hosts 文件:

192.168.1.211 cdh1-master
192.168.1.212 cdh2-slave
192.168.1.213 cdh3-slave

4、关闭防火墙和 selinux(所有节点)

查看防火墙:systemctl status firewalld.service

执行关闭命令:systemctl stop firewalld.service

执行开机禁用防火墙命令:systemctl disable firewalld.service

关闭selinux:vi /etc/sysconfig/selinux 将 SELINUX=enforcing 改为 SELINUX=disabled

5、时钟同步(所有节点)

安装 NTP

yum -y install ntp 

编辑 /etc/ntp.conf 文件添加 NTP 服务器

server 0.cn.pool.ntp.org
server 1.cn.pool.ntp.org
server 2.cn.pool.ntp.org
server 3.cn.pool.ntp.org 

启动 NTP

systemctl enable ntpd
systemctl start ntpd 

手工同步网络时间

ntpdate -u 0.cn.pool.ntp.org 

同步系统时钟

hwclock --systohc

6、配置 ssh 免密登录(所有节点)

在每台机上产生新的rsa公钥私钥文件,并统一拷贝到一个authorized_keys文件中

  1. 登录cdh1-master,在.ssh目录下输入命令:
ssh-keygen -t rsa

三次回车后,该目录下将会产生id_rsa,id_rsa.pub文件。其他主机也使用该方式产生密钥文件。

  1. 登录cdh1-master,将id_rsa.pub公钥内容拷贝到authorized_keys文件中 输入命令:
cat id_rsa.pub >> authorized_keys

赋予权限:

chmod 600 authorized_keys
  1. 登录其他主机,将其他主机的公钥文件内容都拷贝到cdh1-master主机上的authorized_keys文件中,命令如下:
ssh-copy-id -i cdh1-master #登录cdh2-slave,将公钥拷贝到cdh1-master的authorized_keys中
ssh-copy-id -i cdh1-master #登录cdh3-slave,将公钥拷贝到cdh1-master的authorized_keys中 

登录cdh1-master,将授权文件拷贝到cdh2-slave、cdh3-slave…,命令如下:

scp /root/.ssh/authorized_keys cdh2-slave:/root/.ssh/ #拷贝到cdh2-slave上
scp /root/.ssh/authorized_keys cdh3-slave:/root/.ssh/ #拷贝到cdh3-slave上 

7、禁用透明大页面压缩(所有节点)

打开编辑 /etc/rc.local 文件:

vi /etc/rc.local

添加以下内容:

echo never > /sys/kernel/mm/transparent_hugepage/defrag
echo never > /sys/kernel/mm/transparent_hugepage/enabled

8、优化交换分区(所有节点)

RHEL6
编辑 /etc/sysctl.conf 并添加 vm.swappiness=0

vi /etc/sysctl.conf

RHEL7
tuned 服务会动态调整系统参数,查找 tuned 中配置,直接将配置修改为 vm.swappiness=0

cd /usr/lib/tuned/
grep "vm.swappiness" * -R

9、安装jdk(所有节点)

yum -y install oracle-j2sdk1.8-1.8.0+update141-1.x86_64.rpm

/etc/profile 末尾添加以下内容

#set jdk environment
export JAVA_HOME=/usr/java/jdk1.8.0_141-cloudera
export CLASSPATH=.:$JAVA_HOME/lib:$JAVA_HOME/jre/lib:$CLASSPATH
export PATH=$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$PATH

10、安装 mysql(master)

wget -i -c http://dev.mysql.com/get/mysql57-community-release-el7-10.noarch.rpm
yum -y install mysql57-community-release-el7-10.noarch.rpm
yum -y install mysql-community-server
systemctl start mysqld
systemctl status mysqld
grep "password" /var/log/mysqld.log
mysql -u root -p

登录后修改密码(尽量复杂一点,太简单了验证不通过,也可以查看如何关闭mysql校验规则)

ALTER USER 'root'@'localhost' IDENTIFIED BY 'Abc123!@#';

修改mysql外网访问:
增加允许远程访问的用户或者允许现有用户的远程访问。
给root授予在任意主机(%)访问任意数据库的所有权限。

use mysql;
update user set host='%' where user='root' and host='localhost';
exit
systemctl restart mysqld

启动服务并设置开机启动:

systemctl enable mysqld

11、安装 mysql JDBC 驱动(master)

wget https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.46.tar.gz
tar zxvf mysql-connector-java-5.1.46.tar.gz
mkdir -p /usr/share/java/
cp mysql-connector-java-5.1.46/mysql-connector-java-5.1.46-bin.jar /usr/share/java/mysql-connector-java.jar

12、为下表中每个服务创建 utf8 字符集的数据库:

Servicedatabaseuser
Cloudera Manager Serverscmscm
Activity Monitoramonamon
Reports Managerrmanrman
Huehuehue
Hive Metastore Servermetastorehive
Sentry Serversentrysentry
Cloudera Navigator Audit Servernavnav
Cloudera Navigator Metadata Servernavmsnavms
Oozieoozieoozie
CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON scm.* TO 'scm'@'%' IDENTIFIED BY 'Abc123!@#';
CREATE DATABASE amon DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON amon.* TO 'amon'@'%' IDENTIFIED BY 'Abc123!@#';
CREATE DATABASE rman DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON rman.* TO 'rman'@'%' IDENTIFIED BY 'Abc123!@#';
CREATE DATABASE hue DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON hue.* TO 'hue'@'%' IDENTIFIED BY 'Abc123!@#';
CREATE DATABASE metastore DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON metastore.* TO 'hive'@'%' IDENTIFIED BY 'Abc123!@#';
CREATE DATABASE sentry DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON sentry.* TO 'sentry'@'%' IDENTIFIED BY 'Abc123!@#';
CREATE DATABASE nav DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON nav.* TO 'nav'@'%' IDENTIFIED BY 'Abc123!@#';
CREATE DATABASE navms DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON navms.* TO 'navms'@'%' IDENTIFIED BY 'Abc123!@#';
CREATE DATABASE oozie DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON oozie.* TO 'oozie'@'%' IDENTIFIED BY 'Abc123!@#';
flush privileges; 

13、安装 cloudera manager 包

master 节点:

yum -y install cloudera-manager-daemons-6.1.0-769885.el7.x86_64.rpm
yum -y install cloudera-manager-agent-6.1.0-769885.el7.x86_64.rpm
yum -y install cloudera-manager-server-6.1.0-769885.el7.x86_64.rpm

slave 节点:

yum -y install cloudera-manager-daemons-6.1.0-769885.el7.x86_64.rpm
yum -y install cloudera-manager-agent-6.1.0-769885.el7.x86_64.rpm

14、修改 agent 的服务器地址(所有节点)

打开并编辑 /etc/cloudera-scm-agent/config.ini 文件:

/etc/cloudera-scm-agent/config.ini

将 server_host=localhost 改为 master 节点主机名 cdh1-master

15、启动 agent 服务(所有节点)

systemctl start cloudera-scm-agent

16、准备 Cloudera Manager 数据库(master)

/opt/cloudera/cm/schema/scm_prepare_database.sh mysql scm scm

输入scm的密码
若已存在,则执行:

rm /etc/cloudera-scm-server/db.mgmt.properties

17、准备 parcel 包(master)

将已下载的 CDH-6.1.0-1.cdh6.1.0.p0.770702-el7.parcel 和 manifest.json 拷贝至 /opt/cloudera/parcel-repo/ 目录,并且生成 parcel 包的 sha 值:

sha1sum /opt/cloudera/parcel-repo/CDH-6.1.0-1.cdh6.1.0.p0.770702-el7.parcel

将 sha 值填入到 CDH-6.1.0-1.cdh6.1.0.p0.770702-el7.parcel.sha 文件中

vi /opt/cloudera/parcel-repo/CDH-6.1.0-1.cdh6.1.0.p0.770702-el7.parcel.sha

18、启动 Cloudera Manager Server(master)

启动:

systemctl start cloudera-scm-server

19、查看启动日志:

tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log

看到下面的信息,说明启动完成

INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server.

20、启用 Auto-TLS 以自动创建证书(master)

慎用:我用过之后agent连接不了server,导致我装了好多遍,跟证书配置有关系,熟悉可以试一下

JAVA_HOME=/usr/java/jdk1.8.0_141-cloudera /opt/cloudera/cm-agent/bin/certmanager --location /opt/cloudera/CMCA setup --configure-services

21、登录 web 页面(忽略许可授权,选择免费版)

http://192.168.1.211:7180
账号/密码:admin / admin
在这里插入图片描述
在这里插入图片描述
大功告成

GitHub 加速计划 / li / linux-dash
10.39 K
1.2 K
下载
A beautiful web dashboard for Linux
最近提交(Master分支:2 个月前 )
186a802e added ecosystem file for PM2 4 年前
5def40a3 Add host customization support for the NodeJS version 4 年前
Logo

旨在为数千万中国开发者提供一个无缝且高效的云端环境,以支持学习、使用和贡献开源项目。

更多推荐